-- Logs begin at Wed 2022-08-10 16:08:30 UTC, end at Wed 2022-08-10 17:17:08 UTC. -- Aug 10 16:40:49.077789 dv-cinder-836027-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 16:40:49.085286 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 16:40:49.087287 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 16:40:49 2022] *** Aug 10 16:40:49.087287 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 16:40:49.087287 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 16:40:49.087287 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: nodename: dv-cinder-836027-6-iscsi Aug 10 16:40:49.087287 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: machine: x86_64 Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: clock source: unix Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: pcre jit disabled Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: detected number of CPU cores: 4 Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: current working directory: / Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: detected binary path: /usr/bin/uwsgi-core Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: your processes number limit is 47680 Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: your memory page size is 4096 bytes Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: detected max file descriptor number: 2048 Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: lock engine: pthread robust mutexes Aug 10 16:40:49.087618 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: thunder lock: enabled Aug 10 16:40:49.088087 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 16:40:49.088087 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 16:40:49.128813 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: Python main interpreter initialized at 0x56307df6a9c0 Aug 10 16:40:49.128813 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: python threads support enabled Aug 10 16:40:49.128813 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: your server socket listen backlog is limited to 100 connections Aug 10 16:40:49.128813 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: your mercy for graceful operations on workers is 90 seconds Aug 10 16:40:49.129205 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 16:40:49.129272 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: *** Operational MODE: preforking *** Aug 10 16:40:49.129546 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: *** uWSGI is running in multiple interpreter mode *** Aug 10 16:40:49.129546 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: spawned uWSGI master process (pid: 81525) Aug 10 16:40:49.129793 dv-cinder-836027-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 16:40:49.136202 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: spawned uWSGI worker 1 (pid: 81539, cores: 1) Aug 10 16:40:49.136202 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: spawned uWSGI worker 2 (pid: 81540, cores: 1) Aug 10 16:40:49.136202 dv-cinder-836027-6-iscsi devstack@keystone.service[81525]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 16:40:54.630050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:40:54.630050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: from cryptography.utils import int_from_bytes Aug 10 16:40:54.630829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:40:54.630829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: from cryptography.utils import int_from_bytes Aug 10 16:40:54.631792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:40:54.631792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: from cryptography.utils import int_from_bytes Aug 10 16:40:54.632270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:40:54.632270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: from cryptography.utils import int_from_bytes Aug 10 16:40:54.679808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 16:40:54.680013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 16:40:54.680236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] command line args: [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 16:40:54.680512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 16:40:54.680690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ================================================================================ {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 16:40:54.680945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] admin_token = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.681191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] config_dir = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.681273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 16:40:54.681379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.681454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 16:40:54.681560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] config_source = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.681641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] command line args: [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 16:40:54.681740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] debug = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.681863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 16:40:54.681943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.682189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ================================================================================ {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 16:40:54.682343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.682494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] admin_token = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.682638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.682781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.682896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] config_dir = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.682996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] config_source = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] list_limit = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] debug = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] log_config_append = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.683980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] log_dir = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] log_file = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.684903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] list_limit = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] log_config_append = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] log_dir = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] log_file = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.685952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.686925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.687050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.687131 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.687293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.687367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.687567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.687647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.687867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] publish_errors = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.688990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.689136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.689277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.689372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.689567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.689647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] standard_threads = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.689825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.689948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] use_journal = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] use_json = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] publish_errors = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] use_stderr = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.690944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.691085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] use_syslog = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.691164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.691232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.691383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.691457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.691647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.691720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.691909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.691983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.692168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.692256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] standard_threads = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.692438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.692514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.692680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.692754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.692913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.692992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.693095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] use_journal = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.693176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.693275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] use_json = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.693439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.693513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] use_stderr = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.693618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] use_syslog = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.693767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.693890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:40:54.694097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.694170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.694408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.694483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.694722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.694797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.695023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.695106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.695337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.695411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.695640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.695721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.695941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.696016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.696288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.696385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.696603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.696684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.696934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.697102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.697251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.697456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.697608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.697707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.697962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.698035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.698206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.698302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.698462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.698646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.698720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.698947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.699020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.699204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.699288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.699468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.699549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.700248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.700392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.700503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.700734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.700810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.701841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.702962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.703976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.704960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705665 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.705893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.706873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.707954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708286 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.708983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709151 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.709932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.710918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.711856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.712866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.713995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.714967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.715042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.715208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.715294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.715990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.716953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.717980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.718930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] auth.external = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.719873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] auth.password = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] auth.token = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.720908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] auth.external = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.721853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722099 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] auth.password = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] auth.token = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] credential.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.722982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.723951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] credential.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.724973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.725962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.726958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.727935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.728955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.729932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.730965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.731993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.732977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.733965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.734925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.735884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.736893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.737846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738015 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.738967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.739927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.740992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.741937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.742929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.743927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.744896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.745866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.746030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.746107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.746263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.746339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.746504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.747978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748907 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.748978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.749930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.750970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.751962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752477 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.752954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] role.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.753914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] role.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.754879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] role.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.755866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] role.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.756855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.757937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.758960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.759069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.759211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.759313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.759463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.759546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.759799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.759874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.760962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.761105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.761216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.761363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.761459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.761625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.761721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.761879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.762104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.762181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.762906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.762992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.763965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.764981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765141 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.765946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.766943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.767914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.connection = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.768922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.769874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.connection = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.770842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.771970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.772046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.772235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.772316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.772398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 16:40:54.772471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.772710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.772966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.773210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.773462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.773699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.773945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.774185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:40:54.774358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 16:40:54.801386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.801615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.801840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.802172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.802409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.802636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.812410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.813214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.841265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.841493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.841877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.842112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.893632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.893886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.945495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.945697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.948269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:54.948508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.011314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.011547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.011747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.011869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.012915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.035979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.036220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.036345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.036463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.036633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.036726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.036861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.036951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.037103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.037267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.037350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.037503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.037676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.037936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.049214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.049544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.062307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.062546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.062674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.062822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.062948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.063941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:40:55.080420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.080632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:40:55.080900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.081038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.081158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:40:55.081305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:40:55.081425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.081544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.081666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:40:55.081786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.081919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.082052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.082171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.082302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.082420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.082649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.094289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.094576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.094667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.094852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.094936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.095123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.095205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.095411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.102127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.102345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.105243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.105430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.105514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.105715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.105805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.106018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.106143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.106892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.106974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.107158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.107239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.107420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.107502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.107689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.107769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.107961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.108081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.108256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.108339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.108530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.108609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.108797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.131580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.131730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.133100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.133218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.133389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.133529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.138570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.138674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.138870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.139034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.139159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.139408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.145856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.146144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.146261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.146442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.150304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.150556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.150677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.150850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.150967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.151137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.151255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.151416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.151531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.151689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.151805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.151980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.152096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.152277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.152390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.152556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.169007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.169321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.169520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.173230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.173426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.176444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.176602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.176742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.176899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.181177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.181315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.184661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.184938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.187824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.188125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.188298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.188424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.188545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.188712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.188829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.188966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.189082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.189257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.189374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.189590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.205879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.206124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.206244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.206401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.206519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.206661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.206781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.207099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.214007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.214427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.216174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.216444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.216647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.216926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.219273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.219562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.219722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.219842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.220008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.220125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.220421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.220556 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.220725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.221005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.231939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.232251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.232478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.232605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.232764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.232849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.233048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.233140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.233331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.233415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.233606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.233686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.233880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.234157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.250461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.250854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.253482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.253841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.256474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.256805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.258076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.258382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.259586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.259875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.259960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.260171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.260267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.260478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.267185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.267411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.268842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.269068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.271787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.272001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.272087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.272317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.272402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.272607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.272698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.272884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.281447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.281638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.281752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.281915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.282050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.282272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.288891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.289181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.289304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:40:55.289479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.289611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.289732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.289853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.289974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.290094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.290230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.290354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.290516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.290635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.290793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.290922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.291041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.291164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.291605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.291720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.291843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.291964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.292086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.292254 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.292485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.292603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.292773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.293037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:40:55.326687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56307df6a9c0 pid: 81540 (default app) Aug 10 16:40:55.327052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56307df6a9c0 pid: 81539 (default app) Aug 10 16:40:55.330131 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f31be-566e-42b5-bd0a-4ad0b6daef12 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:40:55.330420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f31be-566e-42b5-bd0a-4ad0b6daef12 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:40:55.330694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f31be-566e-42b5-bd0a-4ad0b6daef12 None None] PATH_INFO: `/v3` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:40:55.335064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Wed Aug 10 16:40:55 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:03.125019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2d971-143e-4de6-9c7e-a15eb82eb32f None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:03.125609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2d971-143e-4de6-9c7e-a15eb82eb32f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:03.125609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2d971-143e-4de6-9c7e-a15eb82eb32f None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:03.130139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:03.135858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:03.136367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:03.136460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:03.206996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: INFO dbcounter [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] Registered counter for database keystone Aug 10 16:41:03.225973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG oslo_db.sqlalchemy.engines [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81540) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 16:41:03.232098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writer thread running {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 16:41:03.412892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG passlib.handlers.bcrypt [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81540) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 16:41:03.413407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG passlib.handlers.bcrypt [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81540) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 16:41:03.469229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:03.470837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:03.603129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 16:41:03.776959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2d971-143e-4de6-9c7e-a15eb82eb32f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:03.846252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: INFO dbcounter [None req-24d2d971-143e-4de6-9c7e-a15eb82eb32f None None] Registered counter for database keystone Aug 10 16:41:03.867731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG oslo_db.sqlalchemy.engines [None req-24d2d971-143e-4de6-9c7e-a15eb82eb32f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81539) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 16:41:03.874034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writer thread running {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 16:41:03.991939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2d971-143e-4de6-9c7e-a15eb82eb32f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:03.996270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb28b26-13e9-4ab7-b635-1587a3992f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:03.996905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb28b26-13e9-4ab7-b635-1587a3992f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:03.997187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb28b26-13e9-4ab7-b635-1587a3992f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:03.997459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb28b26-13e9-4ab7-b635-1587a3992f9a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:04.005922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb28b26-13e9-4ab7-b635-1587a3992f9a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:04.059019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ecb28b26-13e9-4ab7-b635-1587a3992f9a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 16:41:04.059284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb28b26-13e9-4ab7-b635-1587a3992f9a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:04.069840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Wed Aug 10 16:41:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:41:04.075911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:04.090512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47ecf-c21a-4084-96e7-7ad162a7495a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:04.095508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51bd35-c0bc-42ac-9d8d-ccdfbbe5c774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:04.096653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51bd35-c0bc-42ac-9d8d-ccdfbbe5c774 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:04.096941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51bd35-c0bc-42ac-9d8d-ccdfbbe5c774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:04.097216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51bd35-c0bc-42ac-9d8d-ccdfbbe5c774 admin admin] PATH_INFO: `/v3/projects/6c8b317543504cf3bafa443bc2186b42` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:04.098664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51bd35-c0bc-42ac-9d8d-ccdfbbe5c774 admin admin] RBAC: Authorizing `identity:get_project(project_id=6c8b317543504cf3bafa443bc2186b42)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:04.149350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9e51bd35-c0bc-42ac-9d8d-ccdfbbe5c774 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 16:41:04.149968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51bd35-c0bc-42ac-9d8d-ccdfbbe5c774 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:04.155114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Aug 10 16:41:04 2022] GET /identity/v3/projects/6c8b317543504cf3bafa443bc2186b42 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:06.160686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7d96b-4762-4676-8729-e9bbb167fbf4 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:06.161255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7d96b-4762-4676-8729-e9bbb167fbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:06.161255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7d96b-4762-4676-8729-e9bbb167fbf4 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:06.165502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:06.171958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983f760-91e3-4555-bafc-83ae096f51f5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:06.172512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983f760-91e3-4555-bafc-83ae096f51f5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:06.172647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983f760-91e3-4555-bafc-83ae096f51f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:06.193971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2983f760-91e3-4555-bafc-83ae096f51f5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:06.195505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2983f760-91e3-4555-bafc-83ae096f51f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:06.203394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 16:41:06.226083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7d96b-4762-4676-8729-e9bbb167fbf4 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:06.239086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7d96b-4762-4676-8729-e9bbb167fbf4 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:06.243455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7735eec-3c4e-4779-b4aa-8f1a020e461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:06.244077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7735eec-3c4e-4779-b4aa-8f1a020e461e admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:06.244395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7735eec-3c4e-4779-b4aa-8f1a020e461e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:06.244676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7735eec-3c4e-4779-b4aa-8f1a020e461e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:06.253084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7735eec-3c4e-4779-b4aa-8f1a020e461e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:06.254894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7735eec-3c4e-4779-b4aa-8f1a020e461e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:06.264212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Wed Aug 10 16:41:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:41:06.270248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983f760-91e3-4555-bafc-83ae096f51f5 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:06.280054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983f760-91e3-4555-bafc-83ae096f51f5 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:06.284544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f07d9-52e6-4ee7-af48-f9d2e071b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:06.285192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f07d9-52e6-4ee7-af48-f9d2e071b782 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:06.285475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f07d9-52e6-4ee7-af48-f9d2e071b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:06.285749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f07d9-52e6-4ee7-af48-f9d2e071b782 admin admin] PATH_INFO: `/v3/users/b68f518c53f84700bf1f6ef1bd005525` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:06.287088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f07d9-52e6-4ee7-af48-f9d2e071b782 admin admin] RBAC: Authorizing `identity:get_user(user_id=b68f518c53f84700bf1f6ef1bd005525)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:06.288978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f07d9-52e6-4ee7-af48-f9d2e071b782 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:06.290535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Wed Aug 10 16:41:06 2022] GET /identity/v3/users/b68f518c53f84700bf1f6ef1bd005525 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:08.380656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9941102-c3f3-44df-80cf-0e3ee4b03ad4 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:08.381288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9941102-c3f3-44df-80cf-0e3ee4b03ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:08.381288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9941102-c3f3-44df-80cf-0e3ee4b03ad4 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:08.382582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:08.387933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26cb4-c076-4128-a92e-1800c6988a0c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:08.388439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26cb4-c076-4128-a92e-1800c6988a0c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:08.388695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26cb4-c076-4128-a92e-1800c6988a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:08.407334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-71b26cb4-c076-4128-a92e-1800c6988a0c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:08.408575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-71b26cb4-c076-4128-a92e-1800c6988a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:08.431398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb96dd-3cb8-4678-a2ac-d09350328e79 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:08.431686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb96dd-3cb8-4678-a2ac-d09350328e79 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:08.432164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb96dd-3cb8-4678-a2ac-d09350328e79 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:08.433506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:08.438880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:08.439170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:08.439452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:08.483532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG passlib.handlers.bcrypt [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81539) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 16:41:08.483962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG passlib.handlers.bcrypt [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81539) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 16:41:08.493261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:08.516453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b26cb4-c076-4128-a92e-1800c6988a0c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:08.530584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:08.532237 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:08.540879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:08.626784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b26cb4-c076-4128-a92e-1800c6988a0c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:08.632766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:08.654994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:08.664201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:08.665157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:08.665581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:08.665999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:08.682270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:08.696645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:08.697526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:08.698330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:08.704731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:08.759116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b0192-b46d-475d-b217-041e93875dbc None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:08.761647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:08.790750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:08.803757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:08.804720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:08.805013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:08.805288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:08.813941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09aa61-f4fb-45d8-9621-e115c7f5b29b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:08.849259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:08.849865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:08.850145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:08.850418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:08.851090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:08.851388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:08.852827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:08.864531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:08.870202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:08.877715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:08.879478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:08.985026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:41:08.991028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:08.994425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:09.083164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d3cc-0406-4429-934a-b8ce4b1112bb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:09.099354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ce3c-ef94-4a06-8616-06fce61e5da5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:09.118788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:09.119428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:09.119709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:09.119977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] PATH_INFO: `/v3/users/b68f518c53f84700bf1f6ef1bd005525` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:09.121501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] RBAC: Authorizing `identity:get_user(user_id=b68f518c53f84700bf1f6ef1bd005525)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:09.123911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:09.125729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:08 2022] GET /identity/v3/users/b68f518c53f84700bf1f6ef1bd005525 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:09.131398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:09.134658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:09.135509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:09.135787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:09.136063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:09.147652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:09.156161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:09.157738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:41:09.177298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:09.178329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:41:09.179294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:41:09.179294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:41:09.179294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:09.179294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:09.186328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:09.240252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8e911-7581-4c1c-b5a1-c96aaf5a38bb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:09.245306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:09.275901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:09.276562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:09.276843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:09.277115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:09.278317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:09.279792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:09.281472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:09.287196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:09.291008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f4c84-37b0-409c-8eb9-53a710941805 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:09.325483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30a229-e35d-47bb-9fc8-677dd341e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:09.326102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30a229-e35d-47bb-9fc8-677dd341e5df None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:09.326946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30a229-e35d-47bb-9fc8-677dd341e5df None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:09.327221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30a229-e35d-47bb-9fc8-677dd341e5df None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:09.327871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30a229-e35d-47bb-9fc8-677dd341e5df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:09.328201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30a229-e35d-47bb-9fc8-677dd341e5df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:09.329632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30a229-e35d-47bb-9fc8-677dd341e5df None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:09.347257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:09.387911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:09.421308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241143f-bc88-4e7c-9f1b-3aa12265afae None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:09.462927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146cfad-2751-493e-9e9a-17fc14a4ac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:09.463563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146cfad-2751-493e-9e9a-17fc14a4ac32 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:09.463848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146cfad-2751-493e-9e9a-17fc14a4ac32 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:09.464125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146cfad-2751-493e-9e9a-17fc14a4ac32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:09.464899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146cfad-2751-493e-9e9a-17fc14a4ac32 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.domain.id=default, user.id=b68f518c53f84700bf1f6ef1bd005525) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:09.465208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146cfad-2751-493e-9e9a-17fc14a4ac32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:09.466784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146cfad-2751-493e-9e9a-17fc14a4ac32 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:09.474752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 11/21] 10.222.0.25 () {56 vars in 1322 bytes} [Wed Aug 10 16:41:09 2022] GET /identity/v3/role_assignments?user.id=b68f518c53f84700bf1f6ef1bd005525&scope.domain.id=default&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 233 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:11.910544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d1c99-4dca-4188-bf7b-153fb4f59e50 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:11.911547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d1c99-4dca-4188-bf7b-153fb4f59e50 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:11.912034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d1c99-4dca-4188-bf7b-153fb4f59e50 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:11.913633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:11.919155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c2a8f-fb73-4fe5-a849-de334d505fa6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:11.919797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c2a8f-fb73-4fe5-a849-de334d505fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:11.919797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c2a8f-fb73-4fe5-a849-de334d505fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:11.949180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-532c2a8f-fb73-4fe5-a849-de334d505fa6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:11.950671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-532c2a8f-fb73-4fe5-a849-de334d505fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:12.039226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:12.105768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d1c99-4dca-4188-bf7b-153fb4f59e50 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:12.221218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee403b1-0931-40fe-b8d0-ae28b98bb76c None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:12.221524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee403b1-0931-40fe-b8d0-ae28b98bb76c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:12.222511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee403b1-0931-40fe-b8d0-ae28b98bb76c None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:12.224090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:12.228555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81482363-ad4d-4e45-8a54-8cdda806b3e2 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:12.229081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81482363-ad4d-4e45-8a54-8cdda806b3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:12.229561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81482363-ad4d-4e45-8a54-8cdda806b3e2 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:12.231749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 14/25] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:12.239740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3aa8-8386-472e-8db9-92ee24ad77f2 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:12.240629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3aa8-8386-472e-8db9-92ee24ad77f2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:12.241110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3aa8-8386-472e-8db9-92ee24ad77f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:12.275859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-47df3aa8-8386-472e-8db9-92ee24ad77f2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:12.278001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-47df3aa8-8386-472e-8db9-92ee24ad77f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:12.405189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 15/26] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:12.407805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae442c-fc32-4e05-bfee-72f5e410a4d3 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:12.408283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae442c-fc32-4e05-bfee-72f5e410a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:12.408720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae442c-fc32-4e05-bfee-72f5e410a4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:12.443961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:12.459770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d1c99-4dca-4188-bf7b-153fb4f59e50 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:12.482933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-22ae442c-fc32-4e05-bfee-72f5e410a4d3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:12.484829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-22ae442c-fc32-4e05-bfee-72f5e410a4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:12.522711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:12.566558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:12.567656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:12.568132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:12.568623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:12.580805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:12.582750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:41:12.707291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:12.708211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:41:12.709022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:41:12.709022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:41:12.709022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:12.710892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 12/27] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 606 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:12.711957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:12.778606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 16/28] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:12.780100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae442c-fc32-4e05-bfee-72f5e410a4d3 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:12.823414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfec18-a202-43c3-8599-5e29830b2526 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:12.861855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:12.862875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:12.863373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:12.863843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:12.877343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:12.884476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:12.885312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:12.886050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:12.888581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 13/29] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:12.890196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:12.921589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:13.003990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93c3aa-d631-4866-8863-119554f73c28 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:13.043199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:13.044318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.044804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.045310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.046171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:13.047848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:13.161203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae442c-fc32-4e05-bfee-72f5e410a4d3 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:13.242611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995c510-cc09-4356-be05-a9e26c8a0905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:13.243604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995c510-cc09-4356-be05-a9e26c8a0905 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.244064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995c510-cc09-4356-be05-a9e26c8a0905 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.244545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995c510-cc09-4356-be05-a9e26c8a0905 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.245659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995c510-cc09-4356-be05-a9e26c8a0905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:13.246181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995c510-cc09-4356-be05-a9e26c8a0905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:13.249413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995c510-cc09-4356-be05-a9e26c8a0905 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:13.256457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:13.285611 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 14/30] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 10 16:41:12 2022] POST /identity/v3/roles => generated 225 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:13.287373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:13.314616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 17/31] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 535 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:13.316667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ea0eb-2853-4adb-9595-b6c83f7f83e2 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.317130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ea0eb-2853-4adb-9595-b6c83f7f83e2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.317588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ea0eb-2853-4adb-9595-b6c83f7f83e2 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.319936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 18/32] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:13.321257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ea0eb-2853-4adb-9595-b6c83f7f83e2 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:13.484233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:13.510870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5b95f-ead8-418c-8a17-1799c14a68e8 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:13.543698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ea0eb-2853-4adb-9595-b6c83f7f83e2 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:13.572080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd7efd-9aef-4b9a-abdd-299dbced178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:13.572944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd7efd-9aef-4b9a-abdd-299dbced178b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.573337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd7efd-9aef-4b9a-abdd-299dbced178b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.573761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd7efd-9aef-4b9a-abdd-299dbced178b None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.574700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd7efd-9aef-4b9a-abdd-299dbced178b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:13.575324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd7efd-9aef-4b9a-abdd-299dbced178b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:13.577211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd7efd-9aef-4b9a-abdd-299dbced178b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:13.588340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff080223-eb33-49e6-8efe-c172301b51df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:13.589232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080223-eb33-49e6-8efe-c172301b51df None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.589652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080223-eb33-49e6-8efe-c172301b51df None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.590053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080223-eb33-49e6-8efe-c172301b51df None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.590490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 15/33] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:13.591904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5a628-8503-4191-a948-98d25445f6ed None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.592744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5a628-8503-4191-a948-98d25445f6ed None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.593068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5a628-8503-4191-a948-98d25445f6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.596604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff080223-eb33-49e6-8efe-c172301b51df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:13.598352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff080223-eb33-49e6-8efe-c172301b51df None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:13.613103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4fe5a628-8503-4191-a948-98d25445f6ed None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:13.614805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4fe5a628-8503-4191-a948-98d25445f6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:13.724228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 16/34] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:13.726371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e12c5-2a9a-4ac9-a047-20f5d04faeda None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.726906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e12c5-2a9a-4ac9-a047-20f5d04faeda None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.727367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e12c5-2a9a-4ac9-a047-20f5d04faeda None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.729851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 17/35] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:13.737647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e12c5-2a9a-4ac9-a047-20f5d04faeda None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:13.822373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 19/36] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:41:13 2022] POST /identity/v3/projects => generated 301 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:13.825138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7625cff-33c0-4c4f-9f6b-fee5f8ff08f4 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.825554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7625cff-33c0-4c4f-9f6b-fee5f8ff08f4 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.826012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7625cff-33c0-4c4f-9f6b-fee5f8ff08f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.853427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a7625cff-33c0-4c4f-9f6b-fee5f8ff08f4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:13.856573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a7625cff-33c0-4c4f-9f6b-fee5f8ff08f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:13.881420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:13.916449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e12c5-2a9a-4ac9-a047-20f5d04faeda None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:13.937247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:13.976553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:13.976553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.976553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.977227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] PATH_INFO: `/v3/users/b68f518c53f84700bf1f6ef1bd005525` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.979341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] RBAC: Authorizing `identity:get_user(user_id=b68f518c53f84700bf1f6ef1bd005525)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:13.981559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:13.988772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 18/37] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:13 2022] GET /identity/v3/users/b68f518c53f84700bf1f6ef1bd005525 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:13.988923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 20/38] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:13.988996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:13.989061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ddbf0-0624-48c7-bf8f-858403d567da None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:13.989061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ddbf0-0624-48c7-bf8f-858403d567da None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:13.989061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ddbf0-0624-48c7-bf8f-858403d567da None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:13.989061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 21/39] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:13.993702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ddbf0-0624-48c7-bf8f-858403d567da None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:14.110890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ddbf0-0624-48c7-bf8f-858403d567da None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:14.127163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a60cad-1eb8-4a1d-8f8e-825abc56debe None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:14.148375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab4c92-3f3c-4032-9e37-66a344058021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:14.149372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4c92-3f3c-4032-9e37-66a344058021 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:14.149741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4c92-3f3c-4032-9e37-66a344058021 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:14.150098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4c92-3f3c-4032-9e37-66a344058021 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:14.151343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab4c92-3f3c-4032-9e37-66a344058021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:14.153360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab4c92-3f3c-4032-9e37-66a344058021 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:14.156800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 22/40] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:14.158628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32b5b4-2e08-4795-9a31-fadad8a10136 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:14.159120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32b5b4-2e08-4795-9a31-fadad8a10136 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:14.159598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32b5b4-2e08-4795-9a31-fadad8a10136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:14.171231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:14.171858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:14.172156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:14.172596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:14.173263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:14.174667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:14.177827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ae32b5b4-2e08-4795-9a31-fadad8a10136 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:14.179184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ae32b5b4-2e08-4795-9a31-fadad8a10136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:14.189627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 19/41] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 10 16:41:13 2022] POST /identity/v3/roles => generated 219 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:14.200496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:14.215721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:14.232729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:14.275366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 23/42] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:14.276348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32b5b4-2e08-4795-9a31-fadad8a10136 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:14.335177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670f9b9-493e-4cd3-88bb-3f254be9d2fc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:14.385576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:14.386454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:14.388391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:14.388723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:14.389989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:14.394570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:14.416553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 20/43] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 10 16:41:14 2022] POST /identity/v3/roles => generated 223 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:14.418983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:14.433682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32b5b4-2e08-4795-9a31-fadad8a10136 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:14.500575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:14.501281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:14.501591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:14.501876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] PATH_INFO: `/v3/domains/default/users/b68f518c53f84700bf1f6ef1bd005525/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:14.503882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b68f518c53f84700bf1f6ef1bd005525, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:14.508193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:14.527259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 24/44] 10.222.0.25 () {58 vars in 1361 bytes} [Wed Aug 10 16:41:14 2022] PUT /identity/v3/domains/default/users/b68f518c53f84700bf1f6ef1bd005525/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:41:14.538368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:14.586331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db9f3c-2378-4bc2-b959-2fc67d24833c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:14.626418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7e5ba-f180-404e-8435-2971fc05d2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:14.627086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e5ba-f180-404e-8435-2971fc05d2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:14.627369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e5ba-f180-404e-8435-2971fc05d2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:14.627644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e5ba-f180-404e-8435-2971fc05d2e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:14.628849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7e5ba-f180-404e-8435-2971fc05d2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:14.630360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7e5ba-f180-404e-8435-2971fc05d2e4 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:14.632313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 21/45] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:14.638140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:14.683189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:14.756706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aaa3d-560c-4fc4-b9d7-e31f1be8ffc8 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:14.790894 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154effdf-9d32-45c1-9aca-f2e3d07feb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:14.791517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154effdf-9d32-45c1-9aca-f2e3d07feb90 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:14.791799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154effdf-9d32-45c1-9aca-f2e3d07feb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:14.792075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154effdf-9d32-45c1-9aca-f2e3d07feb90 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:14.792767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154effdf-9d32-45c1-9aca-f2e3d07feb90 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:14.794237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154effdf-9d32-45c1-9aca-f2e3d07feb90 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:14.824674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 25/46] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:41:14 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:16.584203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b32e7-5957-4d21-974a-136af8a39db1 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:16.584703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b32e7-5957-4d21-974a-136af8a39db1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:16.584894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b32e7-5957-4d21-974a-136af8a39db1 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:16.586386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 22/47] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:16.591687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6312118-225e-453b-aa70-fa29cd6d121c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:16.592122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6312118-225e-453b-aa70-fa29cd6d121c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:16.592926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6312118-225e-453b-aa70-fa29cd6d121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:16.611654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b6312118-225e-453b-aa70-fa29cd6d121c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:16.612961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b6312118-225e-453b-aa70-fa29cd6d121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:16.679864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 26/48] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:16.702383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b32e7-5957-4d21-974a-136af8a39db1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:16.810345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b32e7-5957-4d21-974a-136af8a39db1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:16.829737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394359-33e9-44ec-9187-89bcf6a9be13 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:16.830026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394359-33e9-44ec-9187-89bcf6a9be13 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:16.830313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394359-33e9-44ec-9187-89bcf6a9be13 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:16.831866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 27/49] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:16.836729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81487e-3187-4489-a698-33a352f5abba None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:16.837025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81487e-3187-4489-a698-33a352f5abba None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:16.837302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81487e-3187-4489-a698-33a352f5abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:16.852593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:16.853230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:16.853511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:16.853782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:16.857151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:16.860463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4a81487e-3187-4489-a698-33a352f5abba None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:16.862209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4a81487e-3187-4489-a698-33a352f5abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:16.866612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:16.871372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:16.872894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:16.873689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 23/50] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:16.874970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:16.878528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:16.942936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 28/51] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:16.967145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81487e-3187-4489-a698-33a352f5abba None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:16.991746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02535c-39a8-40d8-b617-a2abdc29d91d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:17.027146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:17.027775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:17.028054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:17.028367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:17.029044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:17.029348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:17.031358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:17.039550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 24/52] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:17.045154 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:17.068787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81487e-3187-4489-a698-33a352f5abba None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:17.105703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:17.106140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:17.106801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:17.107076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:17.107350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:17.109750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:17.111913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:17.114044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 29/53] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:17.119986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:17.142553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:17.152698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9156e-674d-4e96-8310-c896372010cc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:17.199006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:17.199731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:17.200074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:17.200510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] PATH_INFO: `/v3/users/b68f518c53f84700bf1f6ef1bd005525` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:17.202436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] RBAC: Authorizing `identity:get_user(user_id=b68f518c53f84700bf1f6ef1bd005525)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:17.204368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:17.206349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 25/54] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:17 2022] GET /identity/v3/users/b68f518c53f84700bf1f6ef1bd005525 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:17.211823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:17.246338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26614f-bad5-4acb-aba2-2145fdfbdee9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:17.282087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:17.282694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:17.282978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:17.283262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:17.283933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:17.285412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:17.313203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 30/55] 10.222.0.25 () {60 vars in 1140 bytes} [Wed Aug 10 16:41:17 2022] POST /identity/v3/projects => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:41:17.318349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb405ce7-b98c-46e5-a194-aac3c6465864 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:17.348197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:17.358666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f9fd0-0891-4a9d-88bb-eb5c9c60b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:17.359325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9fd0-0891-4a9d-88bb-eb5c9c60b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:17.359820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9fd0-0891-4a9d-88bb-eb5c9c60b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:17.360099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9fd0-0891-4a9d-88bb-eb5c9c60b4c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:17.361375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f9fd0-0891-4a9d-88bb-eb5c9c60b4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:17.362873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f9fd0-0891-4a9d-88bb-eb5c9c60b4c5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:17.364790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 26/56] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:17.371203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:17.450438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:17.496217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0425a5a-e063-471a-b629-e1cfa2be768f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:17.543431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fa7bb-7e21-4069-93b2-94a60e7b5c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:17.544044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fa7bb-7e21-4069-93b2-94a60e7b5c29 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:17.544364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fa7bb-7e21-4069-93b2-94a60e7b5c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:17.544638 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fa7bb-7e21-4069-93b2-94a60e7b5c29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:17.545374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fa7bb-7e21-4069-93b2-94a60e7b5c29 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.domain.id=default, user.id=b68f518c53f84700bf1f6ef1bd005525) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:17.545683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fa7bb-7e21-4069-93b2-94a60e7b5c29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:17.547136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fa7bb-7e21-4069-93b2-94a60e7b5c29 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:17.553369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 31/57] 10.222.0.25 () {56 vars in 1322 bytes} [Wed Aug 10 16:41:17 2022] GET /identity/v3/role_assignments?user.id=b68f518c53f84700bf1f6ef1bd005525&scope.domain.id=default&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 530 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:19.336419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b3698-cb77-4c66-a851-7228f510a873 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:19.336959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b3698-cb77-4c66-a851-7228f510a873 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:19.336959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b3698-cb77-4c66-a851-7228f510a873 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:19.338319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 27/58] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:19.343417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba71fd-f08b-4937-97db-0eedbd3e80cd None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:19.343785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba71fd-f08b-4937-97db-0eedbd3e80cd None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:19.343975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba71fd-f08b-4937-97db-0eedbd3e80cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:19.362436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a9ba71fd-f08b-4937-97db-0eedbd3e80cd None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:19.363629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a9ba71fd-f08b-4937-97db-0eedbd3e80cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:19.433401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 32/59] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:19.458429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b3698-cb77-4c66-a851-7228f510a873 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:19.560805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b3698-cb77-4c66-a851-7228f510a873 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:19.594590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c05efe-3fd7-48a6-b7c4-1bd6877375ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:19.595155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c05efe-3fd7-48a6-b7c4-1bd6877375ce None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:19.595440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c05efe-3fd7-48a6-b7c4-1bd6877375ce None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:19.595717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c05efe-3fd7-48a6-b7c4-1bd6877375ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:19.596899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c05efe-3fd7-48a6-b7c4-1bd6877375ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:19.598347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c05efe-3fd7-48a6-b7c4-1bd6877375ce None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:19.599929 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 28/60] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:19.605621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba71fd-f08b-4937-97db-0eedbd3e80cd None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:19.664467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:19.706132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba71fd-f08b-4937-97db-0eedbd3e80cd None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:19.740093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646069b-9b95-476e-80a0-b1408909d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:19.740731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646069b-9b95-476e-80a0-b1408909d41f None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:19.741014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646069b-9b95-476e-80a0-b1408909d41f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:19.741289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646069b-9b95-476e-80a0-b1408909d41f None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:19.741930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646069b-9b95-476e-80a0-b1408909d41f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:19.743391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646069b-9b95-476e-80a0-b1408909d41f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:19.769739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 33/61] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:41:19 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:23.041288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b08a4-7c21-477d-8e76-7d455db6526d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.042336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b08a4-7c21-477d-8e76-7d455db6526d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.042841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b08a4-7c21-477d-8e76-7d455db6526d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.044436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 29/62] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:23.049976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62b293-c9d6-4b12-b7f2-802fa8082543 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.050805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62b293-c9d6-4b12-b7f2-802fa8082543 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.051284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62b293-c9d6-4b12-b7f2-802fa8082543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.077668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5a62b293-c9d6-4b12-b7f2-802fa8082543 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:23.079231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5a62b293-c9d6-4b12-b7f2-802fa8082543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:23.115864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3381a96-2ff7-47fd-9052-f5efe429d04c None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.116525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3381a96-2ff7-47fd-9052-f5efe429d04c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.117007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3381a96-2ff7-47fd-9052-f5efe429d04c None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.118639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 30/63] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:23.125404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd529d9f-bd5c-404e-94fe-9ef626fce4ee None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.125949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd529d9f-bd5c-404e-94fe-9ef626fce4ee None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.126426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd529d9f-bd5c-404e-94fe-9ef626fce4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.149363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-cd529d9f-bd5c-404e-94fe-9ef626fce4ee None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:23.150997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-cd529d9f-bd5c-404e-94fe-9ef626fce4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:23.208832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 34/64] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:23.228571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:23.244844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 31/65] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:23.262524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62b293-c9d6-4b12-b7f2-802fa8082543 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:23.273678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd529d9f-bd5c-404e-94fe-9ef626fce4ee None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:23.361167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:23.417646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd529d9f-bd5c-404e-94fe-9ef626fce4ee None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:23.433822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62b293-c9d6-4b12-b7f2-802fa8082543 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:23.474362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c3a93-cff0-4f55-a44e-a14428f68caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:23.475024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3a93-cff0-4f55-a44e-a14428f68caa None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.475470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3a93-cff0-4f55-a44e-a14428f68caa None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.475886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3a93-cff0-4f55-a44e-a14428f68caa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.485474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ff176-8d45-4c32-8f88-56792f98e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:23.486504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ff176-8d45-4c32-8f88-56792f98e5fd None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.486987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ff176-8d45-4c32-8f88-56792f98e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.487449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ff176-8d45-4c32-8f88-56792f98e5fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.488579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c3a93-cff0-4f55-a44e-a14428f68caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-822c3a93-cff0-4f55-a44e-a14428f68caa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:23.493753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:23.494617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:23.495883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 32/66] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:23.496738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18466f-8f83-49b8-b46f-f976fd0997e9 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.496738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18466f-8f83-49b8-b46f-f976fd0997e9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.496738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18466f-8f83-49b8-b46f-f976fd0997e9 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.503358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 33/67] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:23.504842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18466f-8f83-49b8-b46f-f976fd0997e9 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:23.505083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ff176-8d45-4c32-8f88-56792f98e5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-468ff176-8d45-4c32-8f88-56792f98e5fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:23.510355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:23.511453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:23.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:23.520244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 35/68] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:23.522441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1ebf-c0cb-4731-a8ab-b1d1d2fccd92 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.522907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1ebf-c0cb-4731-a8ab-b1d1d2fccd92 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.523374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1ebf-c0cb-4731-a8ab-b1d1d2fccd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.541045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-23ed1ebf-c0cb-4731-a8ab-b1d1d2fccd92 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:23.542513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-23ed1ebf-c0cb-4731-a8ab-b1d1d2fccd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:23.571170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:23.627900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18466f-8f83-49b8-b46f-f976fd0997e9 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:23.644032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 36/69] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:23.645267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed1ebf-c0cb-4731-a8ab-b1d1d2fccd92 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:23.683226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0613233-0c0b-4c36-a78a-58b5cb0edbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:23.683860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613233-0c0b-4c36-a78a-58b5cb0edbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.684168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613233-0c0b-4c36-a78a-58b5cb0edbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.684453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613233-0c0b-4c36-a78a-58b5cb0edbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.685138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613233-0c0b-4c36-a78a-58b5cb0edbb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:23.685448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613233-0c0b-4c36-a78a-58b5cb0edbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:23.687656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613233-0c0b-4c36-a78a-58b5cb0edbb3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:23.690527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:23.698385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 34/70] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:23.702220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe78f6-dd64-4279-b977-0ff15be33869 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.702520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe78f6-dd64-4279-b977-0ff15be33869 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.702798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe78f6-dd64-4279-b977-0ff15be33869 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.704831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 35/71] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:23.708825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe78f6-dd64-4279-b977-0ff15be33869 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:23.761945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed1ebf-c0cb-4731-a8ab-b1d1d2fccd92 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:23.794903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:23.795516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.795800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.796630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.797300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:23.797601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:23.799096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:23.807830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 37/72] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:23.808708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:23.809371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe78f6-dd64-4279-b977-0ff15be33869 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:23.825292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:23.847285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc0e02-5e78-4d2f-8ad3-d2126889856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:23.848129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc0e02-5e78-4d2f-8ad3-d2126889856d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.848590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc0e02-5e78-4d2f-8ad3-d2126889856d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.848874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc0e02-5e78-4d2f-8ad3-d2126889856d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc0e02-5e78-4d2f-8ad3-d2126889856d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-e1cc0e02-5e78-4d2f-8ad3-d2126889856d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:23.865575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:23.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:23.867140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 36/73] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:23.867861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41156d31-2cea-4330-ad22-7a272243841f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.868171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41156d31-2cea-4330-ad22-7a272243841f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.868425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41156d31-2cea-4330-ad22-7a272243841f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.886070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-41156d31-2cea-4330-ad22-7a272243841f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:23.887202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-41156d31-2cea-4330-ad22-7a272243841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:23.912810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:23.916855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c92e-6622-4544-89f8-d932329b9241 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:23.952576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:23.953172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.953456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.953736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.954444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 37/74] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:23.955549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a21af-3693-40bd-adec-d2323cb56e56 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:23.955866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a21af-3693-40bd-adec-d2323cb56e56 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:23.956137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a21af-3693-40bd-adec-d2323cb56e56 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:23.957415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 38/75] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:23.961437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a21af-3693-40bd-adec-d2323cb56e56 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:23.979379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:23.980913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:23.982597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 38/76] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:23.983366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.063460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a21af-3693-40bd-adec-d2323cb56e56 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.065813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.081914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45bcb-1f19-4178-a68d-6907593194fd None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.098597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b6bf6-459f-4e1c-94e4-e6c97a345e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.099235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b6bf6-459f-4e1c-94e4-e6c97a345e20 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.099518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b6bf6-459f-4e1c-94e4-e6c97a345e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.099789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b6bf6-459f-4e1c-94e4-e6c97a345e20 None admin] PATH_INFO: `/v3/users/b68f518c53f84700bf1f6ef1bd005525` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.100959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b6bf6-459f-4e1c-94e4-e6c97a345e20 None admin] RBAC: Authorizing `identity:get_user(user_id=b68f518c53f84700bf1f6ef1bd005525)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.102406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b6bf6-459f-4e1c-94e4-e6c97a345e20 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.103977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 39/77] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/users/b68f518c53f84700bf1f6ef1bd005525 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:24.105193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df9aae-8dde-4b6e-95d0-ce01d29fcc44 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.105499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df9aae-8dde-4b6e-95d0-ce01d29fcc44 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.105757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df9aae-8dde-4b6e-95d0-ce01d29fcc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.121019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.121639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.121919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.122193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.122861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:24.123166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.123309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d6df9aae-8dde-4b6e-95d0-ce01d29fcc44 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:24.124520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d6df9aae-8dde-4b6e-95d0-ce01d29fcc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:24.124610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.132653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 39/78] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:41:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:24.133712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.153934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.192709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 40/79] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:24.193581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df9aae-8dde-4b6e-95d0-ce01d29fcc44 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.240971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532b008-cf3c-4ed0-82bb-b9ab7245b9ca None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.274547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.275177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.275456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.275731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.276853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.278301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.280034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 40/80] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:24.281308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.301055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df9aae-8dde-4b6e-95d0-ce01d29fcc44 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.302589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.335420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.336022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.336431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.336718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.365992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.382735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063c5c1-b4ac-4f46-b12c-a88938900b76 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.403466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.405048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.418070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.418691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.418976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.419252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.486701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.488237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.507811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 41/81] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:24.508742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.511236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.580641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.601595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 41/82] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:24.607646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.631417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924c0fa-2fdc-410a-b8d0-9195e13092eb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.667421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.667967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.668210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.670973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.670973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.671244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.675658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 42/83] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:24.675658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.726533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954832a-58ba-4fb1-89a9-8d0d8060de66 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.748897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.763924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.764672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.765498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.765891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.780316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:24.785195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:24.786015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:24.787181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 42/84] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:24.788988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.790127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685973a-c538-4e78-afe4-681770e5e8a2 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.836462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.837079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.837360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.838073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.838739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.840657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.844893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:24.868927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 43/85] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:41:24 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:24.869781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.928511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e11d8e-6c6e-44f7-b7ff-f0844b5111f1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:24.971356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:24.972014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:24.972338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:24.972632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:24.973814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:24.975316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:24.977092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 43/86] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:24.977901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:24.991104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dade72-18d4-458e-ac09-491f7d2ba01c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.023409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:25.038088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.038743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.039105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.039326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:25.042459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:25.042459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:25.044380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 44/87] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:25.045463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:25.107165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9de2e-c254-4b84-8efb-9c573dfb4895 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.128350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:25.150326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.150873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.151159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.151441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:25.197308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c03c43-1ad6-4fe3-804c-8f763b7cd9e9 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.241522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:25.243006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:25.245388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.247317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.247771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.248841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:25.249203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:25.249727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:25.251678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:25.264278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 45/88] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:25.268745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:25.347754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:25.372798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 44/89] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:25.373541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:25.389090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:25.413335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf3f4b-cb6f-49f6-b381-9688a66720cb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.464258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.465274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.465691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.466123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:25.467536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=9b3cf885585848eb912f6b79527201eb) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:25.468003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:25.470423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:25.483459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 46/90] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:25 2022] GET /identity/v3/role_assignments?user.id=9b3cf885585848eb912f6b79527201eb&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:25.484719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:25.507692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e99d40-3e42-4cdc-ab07-546d85639611 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.545830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.547181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.547530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.547806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:25.548717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=b68f518c53f84700bf1f6ef1bd005525) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:25.549034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:25.550547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:25.557678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 45/91] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:25 2022] GET /identity/v3/role_assignments?user.id=b68f518c53f84700bf1f6ef1bd005525&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:25.558622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:25.645809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff872f-9817-4224-8943-20126411a2a9 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.700001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:25.743374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.743947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.744251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.744529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:25.745723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:25.747219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:25.749146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 47/92] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:25 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:25.756779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:25.764993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:25.817439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440ab52-eeb5-4005-8a47-c7e47cc171f5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.871698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.872595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.872771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.873047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:25.875873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:25.877403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:25.879483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 46/93] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:25 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:25.885237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:25.937927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a6757-7cf3-4979-8b5f-d18b4a28746e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:25.992165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:25.993119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:25.993895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:25.994352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:26.075435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144ceb1-03ed-43fb-8204-ba0d54b84118 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:26.106470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:26.130766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:26.132668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:26.152655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:26.153309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:26.153554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:26.153831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:26.154567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] RBAC: Adding query filter params (role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=9b3cf885585848eb912f6b79527201eb) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:26.154875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:26.158682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:26.168620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 47/94] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:25 2022] GET /identity/v3/role_assignments?user.id=9b3cf885585848eb912f6b79527201eb&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=491f8de5da624289a3a9b28c155d764c => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:26.271530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 48/95] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:26.280258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:26.302962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:26.421909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90946cfe-5770-4b09-801e-cd0fc39d62f7 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:26.470765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07288981-a2f1-4239-b6ca-7b1ff1f7a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:26.471416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07288981-a2f1-4239-b6ca-7b1ff1f7a065 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:26.471698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07288981-a2f1-4239-b6ca-7b1ff1f7a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:26.471970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07288981-a2f1-4239-b6ca-7b1ff1f7a065 None admin] PATH_INFO: `/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:26.473145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07288981-a2f1-4239-b6ca-7b1ff1f7a065 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b35b8a4fba4c1cb69fd601a34ff329)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:26.474586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07288981-a2f1-4239-b6ca-7b1ff1f7a065 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:26.476237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 48/96] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:26 2022] GET /identity/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:26.482270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:26.607784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:26.622900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cefc0-8b14-40a8-aa21-aebb48c79038 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:26.674845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f005f-2b89-4636-abaf-7702bba933da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:26.675836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f005f-2b89-4636-abaf-7702bba933da None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:26.676399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f005f-2b89-4636-abaf-7702bba933da None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:26.676884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f005f-2b89-4636-abaf-7702bba933da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:26.677834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f005f-2b89-4636-abaf-7702bba933da None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=e5b35b8a4fba4c1cb69fd601a34ff329, user.id=9b3cf885585848eb912f6b79527201eb) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:26.678354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f005f-2b89-4636-abaf-7702bba933da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:26.680006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f005f-2b89-4636-abaf-7702bba933da None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:26.688544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 49/97] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:26 2022] GET /identity/v3/role_assignments?user.id=9b3cf885585848eb912f6b79527201eb&scope.project.id=e5b35b8a4fba4c1cb69fd601a34ff329&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:28.115894 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ddb1-131b-4eb8-a981-9e77163dad37 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.116426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ddb1-131b-4eb8-a981-9e77163dad37 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.116498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ddb1-131b-4eb8-a981-9e77163dad37 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.117978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 49/98] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:28.130266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1fb20-1107-4c0f-bed4-f86144bcad9a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.131274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1fb20-1107-4c0f-bed4-f86144bcad9a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.131852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1fb20-1107-4c0f-bed4-f86144bcad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.152782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f2e1fb20-1107-4c0f-bed4-f86144bcad9a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:28.154311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f2e1fb20-1107-4c0f-bed4-f86144bcad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:28.253018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c9f83-bd02-4c80-b2ef-f6951f423bd0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.253648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c9f83-bd02-4c80-b2ef-f6951f423bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.254153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c9f83-bd02-4c80-b2ef-f6951f423bd0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.255959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 50/99] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:28.266244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca67dc9-932b-4371-8fd0-77e30e2330e3 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.266844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca67dc9-932b-4371-8fd0-77e30e2330e3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.267321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca67dc9-932b-4371-8fd0-77e30e2330e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.295592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4ca67dc9-932b-4371-8fd0-77e30e2330e3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:28.296177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 50/100] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:28.298160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4ca67dc9-932b-4371-8fd0-77e30e2330e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:28.337142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1fb20-1107-4c0f-bed4-f86144bcad9a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:28.453639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 51/101] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:28.484605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:28.490333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca67dc9-932b-4371-8fd0-77e30e2330e3 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:28.504869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1fb20-1107-4c0f-bed4-f86144bcad9a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:28.516513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:28.550387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ba9ba-1da4-48e5-9917-a67c90e1b3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:28.551048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba9ba-1da4-48e5-9917-a67c90e1b3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.551336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba9ba-1da4-48e5-9917-a67c90e1b3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.551614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba9ba-1da4-48e5-9917-a67c90e1b3a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.573416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ba9ba-1da4-48e5-9917-a67c90e1b3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-907ba9ba-1da4-48e5-9917-a67c90e1b3a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:28.578273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:28.579089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:28.579821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:28.582233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 51/102] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:28.584026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca175a-105d-4999-a3e8-bc6005d0a28f None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.584344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca175a-105d-4999-a3e8-bc6005d0a28f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.584624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca175a-105d-4999-a3e8-bc6005d0a28f None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.585938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 52/103] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:28.592871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca175a-105d-4999-a3e8-bc6005d0a28f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:28.654180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca67dc9-932b-4371-8fd0-77e30e2330e3 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:28.694033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8e037-0935-4b25-a2ff-66e13a98d7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:28.694951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8e037-0935-4b25-a2ff-66e13a98d7da None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.695402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8e037-0935-4b25-a2ff-66e13a98d7da None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.696328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8e037-0935-4b25-a2ff-66e13a98d7da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.698049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8e037-0935-4b25-a2ff-66e13a98d7da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:28.699959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8e037-0935-4b25-a2ff-66e13a98d7da None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:28.702778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 52/104] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:28.704078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455051bb-a71a-4adf-8a11-e37a43ad5c9c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.704529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455051bb-a71a-4adf-8a11-e37a43ad5c9c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.704941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455051bb-a71a-4adf-8a11-e37a43ad5c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.728942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-455051bb-a71a-4adf-8a11-e37a43ad5c9c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:28.730889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-455051bb-a71a-4adf-8a11-e37a43ad5c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:28.768507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca175a-105d-4999-a3e8-bc6005d0a28f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:28.806792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c71434-81dd-489c-9bda-a1e89bf8b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:28.807671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c71434-81dd-489c-9bda-a1e89bf8b8ee None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.808105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c71434-81dd-489c-9bda-a1e89bf8b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.808580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c71434-81dd-489c-9bda-a1e89bf8b8ee None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.809403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c71434-81dd-489c-9bda-a1e89bf8b8ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:28.809858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c71434-81dd-489c-9bda-a1e89bf8b8ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:28.810675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 53/105] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:28.811651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c71434-81dd-489c-9bda-a1e89bf8b8ee None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:28.812808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f91aa-4433-4930-a9a6-b1445b6a32fa None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.813194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f91aa-4433-4930-a9a6-b1445b6a32fa None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.813567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f91aa-4433-4930-a9a6-b1445b6a32fa None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.815353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 54/106] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:28.816031 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f91aa-4433-4930-a9a6-b1445b6a32fa None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:28.822394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 53/107] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:28.823946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c0b5e-a57a-48e8-a1dc-fd4be920cdaf None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:28.824478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c0b5e-a57a-48e8-a1dc-fd4be920cdaf None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:28.824976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c0b5e-a57a-48e8-a1dc-fd4be920cdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:28.833013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:28.833538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:28.846233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d25c0b5e-a57a-48e8-a1dc-fd4be920cdaf None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:28.847728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d25c0b5e-a57a-48e8-a1dc-fd4be920cdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:28.932278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 54/108] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:28.933419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c0b5e-a57a-48e8-a1dc-fd4be920cdaf None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:28.989277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f91aa-4433-4930-a9a6-b1445b6a32fa None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.044687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.045589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.046151 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.046462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.047450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:29.049688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:29.059792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c0b5e-a57a-48e8-a1dc-fd4be920cdaf None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.087405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 55/109] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:41:28 2022] POST /identity/v3/projects => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:29.088238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:29.105687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.106721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.107151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.107620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] PATH_INFO: `/v3/users/ff01aadd28724b8cbe4c8f21809548a6` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.111397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:29.130767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=ff01aadd28724b8cbe4c8f21809548a6)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:29.133318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:29.135503 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 55/110] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:28 2022] GET /identity/v3/users/ff01aadd28724b8cbe4c8f21809548a6 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:29.136752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:29.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:29.230737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdf04b-f3ae-49df-8450-4cec582e1191 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.272730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4712c-b9d8-450b-9d07-350385704cd1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.279218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3e143-ab93-435d-aae2-93f05bf4283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.279835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3e143-ab93-435d-aae2-93f05bf4283c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.280601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3e143-ab93-435d-aae2-93f05bf4283c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.281405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3e143-ab93-435d-aae2-93f05bf4283c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.295852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3e143-ab93-435d-aae2-93f05bf4283c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-67f3e143-ab93-435d-aae2-93f05bf4283c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:29.301825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:29.302749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:29.303498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 56/111] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:29.304458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90acc0-4ea2-4768-8fe9-1e2e4fb58c9d None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.304544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90acc0-4ea2-4768-8fe9-1e2e4fb58c9d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.304778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90acc0-4ea2-4768-8fe9-1e2e4fb58c9d None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.306166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 57/112] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:29.306734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90acc0-4ea2-4768-8fe9-1e2e4fb58c9d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:29.334370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.335184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.335763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.336054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.351496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:29.357563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:29.359528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 56/113] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:29.361616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:29.440836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:29.459536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90acc0-4ea2-4768-8fe9-1e2e4fb58c9d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.486317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:29.508575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711fde5-2643-421c-abb1-3879c05644d3 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.508902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.509473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.509753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.510026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.556102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ad694-5987-466e-8e11-5b88bf9ea65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.557168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ad694-5987-466e-8e11-5b88bf9ea65b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.557659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ad694-5987-466e-8e11-5b88bf9ea65b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.558181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ad694-5987-466e-8e11-5b88bf9ea65b None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.559625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ad694-5987-466e-8e11-5b88bf9ea65b None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:29.560179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ad694-5987-466e-8e11-5b88bf9ea65b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:29.562602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ad694-5987-466e-8e11-5b88bf9ea65b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:29.574132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 57/114] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:29.576533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c174cbf-e52d-48f3-a48a-1eb90da8163a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.577029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c174cbf-e52d-48f3-a48a-1eb90da8163a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.577818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c174cbf-e52d-48f3-a48a-1eb90da8163a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.597187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:29.598834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:29.601541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6c174cbf-e52d-48f3-a48a-1eb90da8163a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:29.603967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6c174cbf-e52d-48f3-a48a-1eb90da8163a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:29.682698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 58/115] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:29.689600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c174cbf-e52d-48f3-a48a-1eb90da8163a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:29.726011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 58/116] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:29.726958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:29.748486 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:29.772815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:29.845092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620004d-c5b8-441d-b5a0-0b8e66bc3fe4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.868177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c174cbf-e52d-48f3-a48a-1eb90da8163a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:29.904500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.905485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.906006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.906493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.908035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:29.909726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:29.911669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 59/117] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:29.912689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:29.914609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:29.915486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:29.915892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:29.916345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:29.919099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:29.919584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:29.922870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:29.936120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 59/118] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:29.937852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.004398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.020992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997f84-01c8-46f0-9964-a9d03eab5b38 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.047395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff152f-54ce-4abb-9888-c8a6b7d22143 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.055617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.058773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.059726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.060009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.060381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.071602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:30.076405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:30.077233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:30.078004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 60/119] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:30.078612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.092635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c44ee6-a8c2-46af-9a16-eef6f4c30701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.093795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44ee6-a8c2-46af-9a16-eef6f4c30701 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.094078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44ee6-a8c2-46af-9a16-eef6f4c30701 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.094352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44ee6-a8c2-46af-9a16-eef6f4c30701 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.095534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c44ee6-a8c2-46af-9a16-eef6f4c30701 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.097037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c44ee6-a8c2-46af-9a16-eef6f4c30701 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.103709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 60/120] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:29 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:30.104688 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2bb2-0758-49b0-b50a-aa87f41d457c None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.104979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2bb2-0758-49b0-b50a-aa87f41d457c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.105365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2bb2-0758-49b0-b50a-aa87f41d457c None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.106516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 61/121] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:30.106516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2bb2-0758-49b0-b50a-aa87f41d457c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.190746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcb788-a07b-4911-8736-7402d37521d1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.219647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2bb2-0758-49b0-b50a-aa87f41d457c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.238149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.239160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.239581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.239999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.242021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.247015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 61/122] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:30.248528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.263681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.264641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.265078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.265485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] PATH_INFO: `/v3/users/b68f518c53f84700bf1f6ef1bd005525` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.267107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] RBAC: Authorizing `identity:get_user(user_id=b68f518c53f84700bf1f6ef1bd005525)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.267343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.269833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.272466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 62/123] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/users/b68f518c53f84700bf1f6ef1bd005525 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:30.273669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.325291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.368406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6835-01c9-4182-8912-47bfbf232cb9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.387228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51579913-5b7d-444b-9e3f-a4e36fd434b1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.405691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481bd52-dd29-4281-be28-58880ac3e8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.406835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481bd52-dd29-4281-be28-58880ac3e8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.407120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481bd52-dd29-4281-be28-58880ac3e8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.409943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481bd52-dd29-4281-be28-58880ac3e8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.409943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481bd52-dd29-4281-be28-58880ac3e8a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:30.409943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481bd52-dd29-4281-be28-58880ac3e8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.410746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481bd52-dd29-4281-be28-58880ac3e8a8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.430336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 62/124] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:30.430336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ed645-e557-4442-8ded-29fe811fc79d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.430336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ed645-e557-4442-8ded-29fe811fc79d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.430336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ed645-e557-4442-8ded-29fe811fc79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.433972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.434599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.434979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.435159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.435894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] RBAC: Adding query filter params (role.id=13803fbdb99c469b9c329ce4e9d995d2, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=ff01aadd28724b8cbe4c8f21809548a6) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:30.437053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.437959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.452199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 63/125] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/role_assignments?user.id=ff01aadd28724b8cbe4c8f21809548a6&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=13803fbdb99c469b9c329ce4e9d995d2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:30.452199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.454332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1a4ed645-e557-4442-8ded-29fe811fc79d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:30.455644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1a4ed645-e557-4442-8ded-29fe811fc79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:30.541030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 63/126] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:30.549428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ed645-e557-4442-8ded-29fe811fc79d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.558716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.570843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e6618-1492-4fcf-a9d7-5bba85a54d70 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.579476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.613721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.614883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.615169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.615451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/9b3cf885585848eb912f6b79527201eb/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.618899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=906d21dfa6d74405b8c0b3b5558f5c88, user_id=9b3cf885585848eb912f6b79527201eb, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.620866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.639989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 64/127] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:30 2022] PUT /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/9b3cf885585848eb912f6b79527201eb/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:41:30.641622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.684010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ed645-e557-4442-8ded-29fe811fc79d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.723208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.723922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.724239 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.724523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.725857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.727372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.729579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 64/128] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:30.730868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.775651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7e458-229b-4a2a-8742-7cdcdc51e6a1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.821771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.822526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.822883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.823236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.824963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:30.826973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:30.829716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 65/129] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:30.830295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.830753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.856454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146aece-10d3-4b0c-bb87-1ea26889c832 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:30.899472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:30.930618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:30.931404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:30.931808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:30.932093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:30.946374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:30.951645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:30.952542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:30.953838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 65/130] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:30.954446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:30.990719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1212323-f8c6-42f6-8343-d95751f61ad0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:31.057015 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:31.057726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.058013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.058295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/b68f518c53f84700bf1f6ef1bd005525/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.061185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] RBAC: Authorizing `identity:create_grant(project_id=906d21dfa6d74405b8c0b3b5558f5c88, user_id=b68f518c53f84700bf1f6ef1bd005525, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:31.063505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:31.092612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090b856-4184-4e2f-99da-512b2f385546 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:31.098939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 66/131] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:30 2022] PUT /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/b68f518c53f84700bf1f6ef1bd005525/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:41:31.099876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:31.139942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:31.141272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.141549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.141798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.143225 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:31.144855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:31.146812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 66/132] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:30 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:31.154541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:31.158412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:31.282389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3148e-a35e-4079-9a96-7524c289c29d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:31.323593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:31.324117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:31.324797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.324797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.325098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/9b3cf885585848eb912f6b79527201eb/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.328200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] RBAC: Authorizing `identity:create_grant(project_id=906d21dfa6d74405b8c0b3b5558f5c88, user_id=9b3cf885585848eb912f6b79527201eb, role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:31.329758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:31.340871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73094025-d008-45cd-8b6a-e526d12dbc5b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:31.355450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 67/133] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:31 2022] PUT /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/9b3cf885585848eb912f6b79527201eb/roles/491f8de5da624289a3a9b28c155d764c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:41:31.399337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c84d29-5b76-4b92-8837-aa20b0dd2e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:31.400032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c84d29-5b76-4b92-8837-aa20b0dd2e9b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.400343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c84d29-5b76-4b92-8837-aa20b0dd2e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.400630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c84d29-5b76-4b92-8837-aa20b0dd2e9b None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.401452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c84d29-5b76-4b92-8837-aa20b0dd2e9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:31.401773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c84d29-5b76-4b92-8837-aa20b0dd2e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:31.403267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c84d29-5b76-4b92-8837-aa20b0dd2e9b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:31.415712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 67/134] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:31.419991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:31.503703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:31.554573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eed800-9d03-4732-a1d8-b0e2f1199367 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:31.575137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2c1b-a551-41a1-9297-4b921f2ea957 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.575434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2c1b-a551-41a1-9297-4b921f2ea957 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.575712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2c1b-a551-41a1-9297-4b921f2ea957 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.577402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 68/135] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:31.583301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b3f45-8fd4-4277-88f0-40881307fcff None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.583588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b3f45-8fd4-4277-88f0-40881307fcff None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.583877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b3f45-8fd4-4277-88f0-40881307fcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.606532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6d2b3f45-8fd4-4277-88f0-40881307fcff None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:31.607765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6d2b3f45-8fd4-4277-88f0-40881307fcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:31.608909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:31.610155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.610538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.610820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.612134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:31.613708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:31.615421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 68/136] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:31 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:31.621321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:31.695489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 69/137] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:31.725660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b3f45-8fd4-4277-88f0-40881307fcff None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:31.788704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711f7e0-40cb-4d8c-976c-05304587218b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:31.853340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:31.864498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:31.865150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.865430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.865697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] PATH_INFO: `/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.866905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] RBAC: Authorizing `identity:get_project(project_id=e5b35b8a4fba4c1cb69fd601a34ff329)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:31.868390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:31.874462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 69/138] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:31 2022] GET /identity/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:31.876100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:31.881561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:31.914914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b3f45-8fd4-4277-88f0-40881307fcff None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:31.963957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:31.964581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:31.965052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:31.965165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:31.969468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:31.970942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:31.972775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 70/139] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:31.989658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:32.033828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cfdc5-2303-43ba-82ee-0d1c41c5b22a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:32.073810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a9aa4-41f5-41bd-b29b-39ee4e0c5189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:32.074839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a9aa4-41f5-41bd-b29b-39ee4e0c5189 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:32.075362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a9aa4-41f5-41bd-b29b-39ee4e0c5189 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:32.075954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a9aa4-41f5-41bd-b29b-39ee4e0c5189 None admin] PATH_INFO: `/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329/users/9b3cf885585848eb912f6b79527201eb/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:32.078782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a9aa4-41f5-41bd-b29b-39ee4e0c5189 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5b35b8a4fba4c1cb69fd601a34ff329, user_id=9b3cf885585848eb912f6b79527201eb, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:32.080699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a9aa4-41f5-41bd-b29b-39ee4e0c5189 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:32.097544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 70/140] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:31 2022] PUT /identity/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329/users/9b3cf885585848eb912f6b79527201eb/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:41:32.147300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb942a-053d-4a0c-b5bb-c96e66e2df2a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:32.196457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0d349-fa50-438d-b7c1-a45bc544f876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:32.197098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0d349-fa50-438d-b7c1-a45bc544f876 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:32.197379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0d349-fa50-438d-b7c1-a45bc544f876 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:32.197654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0d349-fa50-438d-b7c1-a45bc544f876 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:32.198334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0d349-fa50-438d-b7c1-a45bc544f876 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:32.199855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0d349-fa50-438d-b7c1-a45bc544f876 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:32.238441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 71/141] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:41:31 2022] POST /identity/v3/users => generated 283 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:33.123616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d744-76e1-4a91-961e-4c6a4663c5ad None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:33.124411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d744-76e1-4a91-961e-4c6a4663c5ad None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:33.124411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d744-76e1-4a91-961e-4c6a4663c5ad None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:33.125855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 71/142] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:33.133788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5970ed7-83fa-492a-8ae1-680a2f453b70 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:33.134729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5970ed7-83fa-492a-8ae1-680a2f453b70 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:33.135225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5970ed7-83fa-492a-8ae1-680a2f453b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:33.156653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f5970ed7-83fa-492a-8ae1-680a2f453b70 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:33.158178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f5970ed7-83fa-492a-8ae1-680a2f453b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:33.161870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:33.247793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 72/143] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:33.270664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d744-76e1-4a91-961e-4c6a4663c5ad None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:33.345349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:33.397100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d744-76e1-4a91-961e-4c6a4663c5ad None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:33.442251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:33.442970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:33.443314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:33.443599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:33.459865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:33.464825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:33.465672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:33.466404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 72/144] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:33.476458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5970ed7-83fa-492a-8ae1-680a2f453b70 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:33.600282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5970ed7-83fa-492a-8ae1-680a2f453b70 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:33.639871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:33.641431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:33.641913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:33.642400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:33.643275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:33.643787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:33.645393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:33.655258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 73/145] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:33.662845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:33.814789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdf5fa-2007-49a7-a52b-8e5b3bc13e15 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:33.836107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:33.860220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd03d91-6525-4697-a1f6-665c8558b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:33.860865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd03d91-6525-4697-a1f6-665c8558b68b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:33.861137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd03d91-6525-4697-a1f6-665c8558b68b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:33.861406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd03d91-6525-4697-a1f6-665c8558b68b None admin] PATH_INFO: `/v3/users/ff01aadd28724b8cbe4c8f21809548a6` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:33.862588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd03d91-6525-4697-a1f6-665c8558b68b None admin] RBAC: Authorizing `identity:get_user(user_id=ff01aadd28724b8cbe4c8f21809548a6)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:33.864065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd03d91-6525-4697-a1f6-665c8558b68b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:33.865753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 73/146] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:33 2022] GET /identity/v3/users/ff01aadd28724b8cbe4c8f21809548a6 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:33.871639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:33.874453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:34.013471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c5484-3801-4ba6-8d92-c493e66a2af5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:34.031352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354e53f-344f-46d3-bde3-7bb159cca95d None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.031643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354e53f-344f-46d3-bde3-7bb159cca95d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.031907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354e53f-344f-46d3-bde3-7bb159cca95d None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.033323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 74/147] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:34.038297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60dea-d367-4ac7-84bd-6d56bf3f5730 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.038583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60dea-d367-4ac7-84bd-6d56bf3f5730 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.038858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60dea-d367-4ac7-84bd-6d56bf3f5730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.065792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c3d60dea-d367-4ac7-84bd-6d56bf3f5730 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:34.066994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c3d60dea-d367-4ac7-84bd-6d56bf3f5730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:34.070497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:34.071141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.071420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.071690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.072850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:34.074291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:34.075921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 74/148] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:33 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:34.082389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:34.185889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 75/149] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:34.200329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:34.210470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d60dea-d367-4ac7-84bd-6d56bf3f5730 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:34.244138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34c11-3b36-4d4b-9293-9944d2e71c87 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:34.305606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:34.306702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.307206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.307686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/ff01aadd28724b8cbe4c8f21809548a6/roles/13803fbdb99c469b9c329ce4e9d995d2` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.314346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=906d21dfa6d74405b8c0b3b5558f5c88, user_id=ff01aadd28724b8cbe4c8f21809548a6, role_id=13803fbdb99c469b9c329ce4e9d995d2)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:34.316047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:34.316170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d60dea-d367-4ac7-84bd-6d56bf3f5730 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:34.334199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:34.350952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 75/150] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:34 2022] PUT /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/users/ff01aadd28724b8cbe4c8f21809548a6/roles/13803fbdb99c469b9c329ce4e9d995d2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:41:34.353365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2ad35-6134-4241-8a71-956bebd4ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:34.354014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2ad35-6134-4241-8a71-956bebd4ca6b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.354291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2ad35-6134-4241-8a71-956bebd4ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.354560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2ad35-6134-4241-8a71-956bebd4ca6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.367701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2ad35-6134-4241-8a71-956bebd4ca6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-7ff2ad35-6134-4241-8a71-956bebd4ca6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:34.372898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:34.373808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:34.374540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 76/151] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:34.381092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:34.449459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe4c5-45f5-4901-9449-9cc96578e268 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.449791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe4c5-45f5-4901-9449-9cc96578e268 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.450025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe4c5-45f5-4901-9449-9cc96578e268 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.451543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 77/152] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:34.456833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fe927-7ba7-4054-9bb8-540d4d87be48 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.457102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fe927-7ba7-4054-9bb8-540d4d87be48 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.457378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fe927-7ba7-4054-9bb8-540d4d87be48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.492634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b87fe927-7ba7-4054-9bb8-540d4d87be48 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:34.493851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b87fe927-7ba7-4054-9bb8-540d4d87be48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:34.548349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50474dce-3efb-464e-9160-23809e5a22d1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:34.604843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:34.611296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 78/153] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:34.612710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362085bc-76e5-41c3-ac98-6f1cb36a00ad None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.612994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362085bc-76e5-41c3-ac98-6f1cb36a00ad None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.613268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362085bc-76e5-41c3-ac98-6f1cb36a00ad None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.617447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 79/154] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:34.617642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:34.618396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.618753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.619104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.619865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:34.620270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:34.621744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:34.628989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82240c95-4efc-415e-b10b-45adbfbe2759 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.629270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82240c95-4efc-415e-b10b-45adbfbe2759 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.629548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82240c95-4efc-415e-b10b-45adbfbe2759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.639948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 76/155] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:34.649396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:34.669649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-82240c95-4efc-415e-b10b-45adbfbe2759 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:34.670872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-82240c95-4efc-415e-b10b-45adbfbe2759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:34.769844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 80/156] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:34.770739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82240c95-4efc-415e-b10b-45adbfbe2759 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:34.821759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:34.852654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef709-b490-43be-ba15-bf9f11783b16 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:34.917482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:34.918062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:34.918336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:34.918611 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:34.945519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82240c95-4efc-415e-b10b-45adbfbe2759 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:34.947567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:34.956204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:34.958483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:34.960483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:34.964757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 77/157] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:34.964953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.009037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.009682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.009960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.010234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] PATH_INFO: `/v3/users/b68f518c53f84700bf1f6ef1bd005525` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.011575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] RBAC: Authorizing `identity:get_user(user_id=b68f518c53f84700bf1f6ef1bd005525)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:35.013100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:35.015199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 81/158] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:34 2022] GET /identity/v3/users/b68f518c53f84700bf1f6ef1bd005525 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:35.015879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.040405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:35.092976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca52384-8f1c-41e4-9ab5-a906aa76af7d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:35.129298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.129945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.130225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.130513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.136627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c86df9-30c0-4f1a-96c3-cd8ae137eb91 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:35.139338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:35.142414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:35.147925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:35.148978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:35.149788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 78/159] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:35.150395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.183965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.184993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.185461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.185917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.188253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:35.188802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:35.190443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:35.199781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 82/160] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:35.200997 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.263487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ded93e-2a47-41e0-8c9c-1c57fb20d175 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:35.288510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:35.300621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.301224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.301498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.301852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.328995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8010f2-168f-4e7f-af27-ca8e716242fe None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:35.374606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:35.375348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:35.385281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.386223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.386706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.387186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.388049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:35.388563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:35.391430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:35.404920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 83/161] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:35.406569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.453182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:35.484953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 79/162] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:35.486317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.524519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:35.577495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d6b47-437b-4883-97e7-4a1bd7f72539 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:35.634160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.634815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.635096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.635368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.636559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:35.638013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:35.639620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 84/163] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:35.640970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.746841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f7383-1fb0-4de6-b704-54d2fecfc3ca None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:35.832664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:35.842623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d8c73-5663-463e-8488-d4bd84421d84 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:35.852927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.853605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.853887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.854159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.855400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:35.856896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:35.858671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 80/164] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:35.860739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:35.909027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:35.909687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:35.910087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:35.910369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:35.911549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:35.913399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:35.915124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 85/165] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:35.915914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:36.084861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:36.123333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6818f0-4981-4067-a7a5-b34b142aab7b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:36.199213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb1b3a-f118-4b17-bb0c-aef6c8624f63 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:36.206667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:36.207360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:36.207642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:36.207911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:36.265918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:36.266551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:36.266835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:36.267107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:36.318072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:36.377860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:36.385594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:36.401562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:36.403062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:36.523077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 86/166] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:36.523987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:36.632610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:36.657249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:36.664204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 81/167] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 805 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:36.665833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:36.738863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfacf4f-c9c1-4f7d-a4b1-3f45da738efc None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:36.819971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc7609-af9b-4da5-81da-4d44097a278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:36.821052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc7609-af9b-4da5-81da-4d44097a278d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:36.821468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc7609-af9b-4da5-81da-4d44097a278d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:36.821886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc7609-af9b-4da5-81da-4d44097a278d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:36.823130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc7609-af9b-4da5-81da-4d44097a278d None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=b68f518c53f84700bf1f6ef1bd005525) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:36.823580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc7609-af9b-4da5-81da-4d44097a278d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:36.825900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc7609-af9b-4da5-81da-4d44097a278d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:36.845476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 87/168] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:36 2022] GET /identity/v3/role_assignments?user.id=b68f518c53f84700bf1f6ef1bd005525&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:36.846856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca5e46-5a97-4620-9e75-8bebde1335bb None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:36.847153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca5e46-5a97-4620-9e75-8bebde1335bb None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:36.847432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca5e46-5a97-4620-9e75-8bebde1335bb None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:36.848775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 88/169] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:36.853530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca5e46-5a97-4620-9e75-8bebde1335bb None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:36.889128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d5aeb-8ce7-4d65-95bb-9d238bf4fda7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:36.959616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208aaaf-8bb0-4599-b6e7-2b1e0b0f76dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:36.960711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208aaaf-8bb0-4599-b6e7-2b1e0b0f76dc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:36.961033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208aaaf-8bb0-4599-b6e7-2b1e0b0f76dc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:36.961324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208aaaf-8bb0-4599-b6e7-2b1e0b0f76dc None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:36.968167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208aaaf-8bb0-4599-b6e7-2b1e0b0f76dc None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:36.968167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208aaaf-8bb0-4599-b6e7-2b1e0b0f76dc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:36.968869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 82/170] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:36 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:36.970225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad902bd-ef61-4859-98a6-fe194b3a0b16 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:36.970518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad902bd-ef61-4859-98a6-fe194b3a0b16 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:36.970799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad902bd-ef61-4859-98a6-fe194b3a0b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.009818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9ad902bd-ef61-4859-98a6-fe194b3a0b16 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:37.011070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9ad902bd-ef61-4859-98a6-fe194b3a0b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:37.100541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:37.112359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca5e46-5a97-4620-9e75-8bebde1335bb None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:37.134917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 83/171] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:37.135960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad902bd-ef61-4859-98a6-fe194b3a0b16 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:37.181326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:37.189771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:37.190568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.190946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.191343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.193033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:37.194619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:37.196955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 89/172] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:36 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:37.197974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:37.337202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad902bd-ef61-4859-98a6-fe194b3a0b16 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:37.403031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:37.404501 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.404785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.405054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.407084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] RBAC: Adding query filter params (role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=9b3cf885585848eb912f6b79527201eb) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:37.407399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:37.410365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:37.426268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 84/173] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:37 2022] GET /identity/v3/role_assignments?user.id=9b3cf885585848eb912f6b79527201eb&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=491f8de5da624289a3a9b28c155d764c => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:37.426469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320856e7-f5eb-4ed1-9d83-48a3e0453d01 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:37.426856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:37.521910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444dab2-6b1c-40c1-b32b-c5c80e515ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:37.522476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444dab2-6b1c-40c1-b32b-c5c80e515ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.522756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444dab2-6b1c-40c1-b32b-c5c80e515ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.523033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444dab2-6b1c-40c1-b32b-c5c80e515ce7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.533670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:37.551336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444dab2-6b1c-40c1-b32b-c5c80e515ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-f444dab2-6b1c-40c1-b32b-c5c80e515ce7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:37.558364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:37.559371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:37.560132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 90/174] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:37.560821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8181a1-6ae2-466b-8ae0-1a2afdc4afd8 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.561716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8181a1-6ae2-466b-8ae0-1a2afdc4afd8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.561716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8181a1-6ae2-466b-8ae0-1a2afdc4afd8 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.562656 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 91/175] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:37.574017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c8cf9-4fec-4818-afb8-ff621ead0c0f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.574333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c8cf9-4fec-4818-afb8-ff621ead0c0f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.574562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c8cf9-4fec-4818-afb8-ff621ead0c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.613550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5e6c8cf9-4fec-4818-afb8-ff621ead0c0f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:37.614892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5e6c8cf9-4fec-4818-afb8-ff621ead0c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:37.629500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a916-5b73-4874-b4ab-05e149b20faa None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:37.674423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:37.675051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.675327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.675597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.677650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=9b3cf885585848eb912f6b79527201eb) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:37.677963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:37.679429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:37.695124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 85/176] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:37 2022] GET /identity/v3/role_assignments?user.id=9b3cf885585848eb912f6b79527201eb&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:37.696217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:37.746109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:37.799633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 92/177] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:37.819901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db79d4-31ae-4e60-a921-5762b90e747a None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.820219 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db79d4-31ae-4e60-a921-5762b90e747a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db79d4-31ae-4e60-a921-5762b90e747a None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.822271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 93/178] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:37.833478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626c917-b532-4bfc-aefb-b1dfd1395ec0 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.833780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626c917-b532-4bfc-aefb-b1dfd1395ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.834035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626c917-b532-4bfc-aefb-b1dfd1395ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.867602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7626c917-b532-4bfc-aefb-b1dfd1395ec0 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:37.868898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7626c917-b532-4bfc-aefb-b1dfd1395ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:37.887504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3852b-78ce-4a35-bed0-a5922a641986 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:37.900891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:37.963064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32b7c6-cd77-4fff-b014-781a55f3c7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:37.963955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b7c6-cd77-4fff-b014-781a55f3c7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.964332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b7c6-cd77-4fff-b014-781a55f3c7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.964628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 94/179] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:37.964915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b7c6-cd77-4fff-b014-781a55f3c7a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.965818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626c917-b532-4bfc-aefb-b1dfd1395ec0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:37.968261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32b7c6-cd77-4fff-b014-781a55f3c7a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:37.968261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32b7c6-cd77-4fff-b014-781a55f3c7a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:37.969217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32b7c6-cd77-4fff-b014-781a55f3c7a0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:37.982887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 86/180] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:37.989676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974ae10-1920-43bc-92b4-76a3cfa29ed3 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.989676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974ae10-1920-43bc-92b4-76a3cfa29ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.989676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974ae10-1920-43bc-92b4-76a3cfa29ed3 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.989676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 87/181] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:37.993219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeaef6-dd5a-46c0-943c-fb353102888f None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:37.993512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeaef6-dd5a-46c0-943c-fb353102888f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:37.993831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeaef6-dd5a-46c0-943c-fb353102888f None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:37.999465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 88/182] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:38.004175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6186aa-708b-4550-8a7e-d94610b3e071 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.004175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6186aa-708b-4550-8a7e-d94610b3e071 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.004175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6186aa-708b-4550-8a7e-d94610b3e071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.044185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4a6186aa-708b-4550-8a7e-d94610b3e071 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:38.044185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4a6186aa-708b-4550-8a7e-d94610b3e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:38.137660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626c917-b532-4bfc-aefb-b1dfd1395ec0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.142466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 89/183] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:38.149084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6186aa-708b-4550-8a7e-d94610b3e071 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.160512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:38.185673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.186329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.186613 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.186889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.197819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:38.202762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:38.204173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 95/184] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:38.210128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.279973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:38.287137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6186aa-708b-4550-8a7e-d94610b3e071 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.330528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb18f3-b297-4358-abec-11c150e6a8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.331189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb18f3-b297-4358-abec-11c150e6a8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.331616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb18f3-b297-4358-abec-11c150e6a8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.331896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb18f3-b297-4358-abec-11c150e6a8b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.332426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb192fd-a542-4b4e-932a-521ee510e38f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.346664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb18f3-b297-4358-abec-11c150e6a8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-89bb18f3-b297-4358-abec-11c150e6a8b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:38.351446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:38.352497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:38.353300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 90/185] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:38.353932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66766c58-8c70-4bd0-bbdc-3c878f5361cf None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.354393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66766c58-8c70-4bd0-bbdc-3c878f5361cf None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.354393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66766c58-8c70-4bd0-bbdc-3c878f5361cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.368689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.369336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.369615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.369887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] PATH_INFO: `/v3/users/9b3cf885585848eb912f6b79527201eb` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.371033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] RBAC: Authorizing `identity:get_user(user_id=9b3cf885585848eb912f6b79527201eb)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:38.372527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:38.372884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-66766c58-8c70-4bd0-bbdc-3c878f5361cf None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:38.374128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 96/186] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/users/9b3cf885585848eb912f6b79527201eb => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:38.379240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-66766c58-8c70-4bd0-bbdc-3c878f5361cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:38.379346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.432817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:38.455326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 91/187] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:38.468705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66766c58-8c70-4bd0-bbdc-3c878f5361cf None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.504928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fe60e-bd33-4dff-a4e4-d024a2258459 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.538314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.538949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.539229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.539500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.549861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:38.554615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:38.555558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 97/188] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:38.561683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.563523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:38.575763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66766c58-8c70-4bd0-bbdc-3c878f5361cf None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.612979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.613593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.613874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.614142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.614803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:38.615103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:38.616522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:38.625645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 92/189] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:38.629144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.664345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd18d12-f94b-4ad4-b544-85039b28f57a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.666677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:38.716346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.717085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.717360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.717723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.718307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:38.718612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:38.719996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:38.728282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 98/190] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:38.729103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.736481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf696-5319-423c-9344-379c2361ef3b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.774715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.775339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.775621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.775897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.794892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:38.847254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:38.848871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:38.849492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fec58-58aa-4397-a877-8b0daba7cfdb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:38.885229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:38.885857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:38.886136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:38.886408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:38.887513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:38.889101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:38.890722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 99/191] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:38.891498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:38.929569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:38.954972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 93/192] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:38.956086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.004075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8993-3d9b-4140-96d4-96b9fab42cb7 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.005223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:39.042703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.043374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.043659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.043926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.044710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:39.045008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.046628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.056124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 100/193] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:39.059702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233d02e-f2a2-4d75-a1e9-f3aaf67100bb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.061026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.093354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.094484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.094769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.095036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] PATH_INFO: `/v3/users/4b23a40791d54b2fb6531c2fccff996f` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.107767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] RBAC: Authorizing `identity:get_user(user_id=4b23a40791d54b2fb6531c2fccff996f)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.109910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.111529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 94/194] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:38 2022] GET /identity/v3/users/4b23a40791d54b2fb6531c2fccff996f => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:39.112301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.166277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:39.173179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3904df-e952-4a38-9e0c-8bee38d0d3f2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.207694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.208287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.208567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.208847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] PATH_INFO: `/v3/users/4b23a40791d54b2fb6531c2fccff996f` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.209977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] RBAC: Authorizing `identity:get_user(user_id=4b23a40791d54b2fb6531c2fccff996f)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.211433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.213054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 101/195] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:39 2022] GET /identity/v3/users/4b23a40791d54b2fb6531c2fccff996f => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:39.213643 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f75c7e-6b7a-4281-92f0-adee56e634cb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.213812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.226412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:39.254754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.255362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.255639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.255908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.256590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.258077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.278598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 95/196] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:41:39 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:39.279488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.332724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a87c95-5872-4243-bafe-4d4e660897f2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.379124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.379832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.380111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.380414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] PATH_INFO: `/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.381764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b35b8a4fba4c1cb69fd601a34ff329)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.383488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.385374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 102/197] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:39 2022] GET /identity/v3/projects/e5b35b8a4fba4c1cb69fd601a34ff329 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:39.386190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.403931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f31bb-14f2-4335-a993-61b6a929eaed None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.435387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:39.452551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.453471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.453908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.454320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] PATH_INFO: `/v3/users/ff01aadd28724b8cbe4c8f21809548a6` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.456785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] RBAC: Authorizing `identity:get_user(user_id=ff01aadd28724b8cbe4c8f21809548a6)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.458354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.459985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 96/198] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:39 2022] GET /identity/v3/users/ff01aadd28724b8cbe4c8f21809548a6 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:39.460831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.499144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:39.529454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f9eb-2e22-4567-bb95-716db028d856 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.582062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.582682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.582959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.583234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.590485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c85c5-054a-4192-9c81-3bfe90b8c865 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.636167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:39.636812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:39.637094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:39.637366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:39.673813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.675783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.710488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:39.739490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:39.741461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:39.780489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:39.818619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 103/199] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:39.820004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.870268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 97/200] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:39.871131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:39.956994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903809ae-08e3-47db-82f1-c0f6cf7296d9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:39.998987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb557188-5738-43c4-8afd-bf56639b0018 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:40.001243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:40.008832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.010234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.010721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.011659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.013072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=e5b35b8a4fba4c1cb69fd601a34ff329, user.id=9b3cf885585848eb912f6b79527201eb) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:40.013738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.015768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.024073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 104/201] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:39 2022] GET /identity/v3/role_assignments?user.id=9b3cf885585848eb912f6b79527201eb&scope.project.id=e5b35b8a4fba4c1cb69fd601a34ff329&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:40.025439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:40.048455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:40.058304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.060420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.062630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.063533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.213624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.215324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.228170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276774f-c47f-4cd9-bc10-689aac187cb3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:40.268325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.268887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.269152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.269424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.270545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.271986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.274134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 105/202] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:40 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:40.274912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:40.341787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 98/203] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:40.342508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:40.368290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:40.394367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:40.423045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8226-ff92-492e-9280-0b339474b863 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:40.467974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.469889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.470305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.470711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.472331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.474211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.476516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 106/204] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:40 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:40.477505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:40.485366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63514da3-d18a-4c34-b7fa-b9aea9c181f5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:40.524788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.526162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.526639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.527102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.528024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=4b23a40791d54b2fb6531c2fccff996f) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:40.528729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.530557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.536811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 99/205] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:40 2022] GET /identity/v3/role_assignments?user.id=4b23a40791d54b2fb6531c2fccff996f&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:40.537870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:40.600673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662854a-6596-4de0-a91c-9e31cf444ec3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:40.635054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:40.643343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.644321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.644816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.645285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.649461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.651738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.654026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 107/206] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:40 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:40.661580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:40.732120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:40.747530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b33a3-f536-41f1-a68b-a98c22b3a727 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:40.807909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c571c-00ba-4664-8351-e12666e3f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.808469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c571c-00ba-4664-8351-e12666e3f83a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.808749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c571c-00ba-4664-8351-e12666e3f83a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.809017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c571c-00ba-4664-8351-e12666e3f83a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.809746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c571c-00ba-4664-8351-e12666e3f83a None admin] RBAC: Adding query filter params (role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=4b23a40791d54b2fb6531c2fccff996f) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:40.810051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c571c-00ba-4664-8351-e12666e3f83a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.811672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c571c-00ba-4664-8351-e12666e3f83a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.819413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 100/207] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:40 2022] GET /identity/v3/role_assignments?user.id=4b23a40791d54b2fb6531c2fccff996f&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=491f8de5da624289a3a9b28c155d764c => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:40.845735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d4d8d-db4c-4dec-a76e-549d8ad65b43 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:40.892877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c6e56-5139-440f-9d76-490c16834db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:40.893879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6e56-5139-440f-9d76-490c16834db1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:40.894362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6e56-5139-440f-9d76-490c16834db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:40.894824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6e56-5139-440f-9d76-490c16834db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:40.895792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c6e56-5139-440f-9d76-490c16834db1 None admin] RBAC: Adding query filter params (role.id=13803fbdb99c469b9c329ce4e9d995d2, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88, user.id=ff01aadd28724b8cbe4c8f21809548a6) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:40.896349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c6e56-5139-440f-9d76-490c16834db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:40.898030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c6e56-5139-440f-9d76-490c16834db1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:40.905463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 108/208] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:40 2022] GET /identity/v3/role_assignments?user.id=ff01aadd28724b8cbe4c8f21809548a6&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=13803fbdb99c469b9c329ce4e9d995d2 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:41.520673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae705f9-82eb-4656-a138-ff18302c80ef None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:41.521057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae705f9-82eb-4656-a138-ff18302c80ef None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:41.521241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae705f9-82eb-4656-a138-ff18302c80ef None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:41.522734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 101/209] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:41.528835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ce1a8-a9c3-4c86-af66-ae9a9046dc46 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:41.529717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ce1a8-a9c3-4c86-af66-ae9a9046dc46 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:41.530231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ce1a8-a9c3-4c86-af66-ae9a9046dc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:41.549789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-157ce1a8-a9c3-4c86-af66-ae9a9046dc46 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:41.551284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-157ce1a8-a9c3-4c86-af66-ae9a9046dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:41.617611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:41.624131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6be86-416b-4329-9ced-8b3052f47288 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:41.624669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6be86-416b-4329-9ced-8b3052f47288 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:41.624783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6be86-416b-4329-9ced-8b3052f47288 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:41.626365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 102/210] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:41.627615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 109/211] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:41.631480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11f454-ff81-48d3-bc06-6e7e6178ff3f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:41.631713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11f454-ff81-48d3-bc06-6e7e6178ff3f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:41.631992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11f454-ff81-48d3-bc06-6e7e6178ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:41.650564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5b11f454-ff81-48d3-bc06-6e7e6178ff3f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:41.651545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ce1a8-a9c3-4c86-af66-ae9a9046dc46 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:41.651765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5b11f454-ff81-48d3-bc06-6e7e6178ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:41.735726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 103/212] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:41.759881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11f454-ff81-48d3-bc06-6e7e6178ff3f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:41.769494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ce1a8-a9c3-4c86-af66-ae9a9046dc46 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:41.814007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:41.814700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:41.815024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:41.815294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:41.826584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:41.832269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:41.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:41.833859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:41.840854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 110/213] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:41.843323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:41.862870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:41.879088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11f454-ff81-48d3-bc06-6e7e6178ff3f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:41.912631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:41.928890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:41.929537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:41.929817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:41.930085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:41.931251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:41.932732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:41.934430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 104/214] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:41.943287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:41.980838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1418-1e18-4de6-9ed4-584dd598bc7c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:42.027870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:42.028602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:42.028981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:42.029352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:42.030648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:42.031960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:42.033925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:42.051400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 111/215] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:42.061170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:42.082306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5138e3-c836-4ecf-b6a6-395d862101f1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:42.119889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:42.120754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:42.121099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:42.121447 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:42.122310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:42.123927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:42.150957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 105/216] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:41:41 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:41:42.201269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba07fb3-2b29-49c1-ba37-2b6a1bcba088 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:42.221488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:42.243794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:42.244557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:42.244844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:42.245122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] PATH_INFO: `/v3/users/7db7471fa61b4255868f9a7d8217e800` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:42.262070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] RBAC: Authorizing `identity:get_user(user_id=7db7471fa61b4255868f9a7d8217e800)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:42.263662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:42.266445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 112/217] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:42 2022] GET /identity/v3/users/7db7471fa61b4255868f9a7d8217e800 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:42.273396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:42.291383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:42.390182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d575f5-9a5f-4635-9254-33099c6728b3 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:42.434238 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:42.434873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:42.435161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:42.435437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:42.511054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:42.512562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:42.528124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:42.628516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 106/218] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:42.630535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:42.746617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1097e7-0900-4f28-b05c-463c3c63516e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:42.794293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569b061-ccea-46be-8eec-ff8d96a28d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:42.794946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569b061-ccea-46be-8eec-ff8d96a28d9a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:42.795227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569b061-ccea-46be-8eec-ff8d96a28d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:42.795501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569b061-ccea-46be-8eec-ff8d96a28d9a None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:42.796707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569b061-ccea-46be-8eec-ff8d96a28d9a None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:42.798750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569b061-ccea-46be-8eec-ff8d96a28d9a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:42.800459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 113/219] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:42 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:42.806521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:42.972348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947d01-4b5d-4e42-b3f2-8cf7e3b53543 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:42.972742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947d01-4b5d-4e42-b3f2-8cf7e3b53543 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:42.972917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947d01-4b5d-4e42-b3f2-8cf7e3b53543 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:42.974266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 114/220] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:42.979329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eaff9-a3d9-430a-9fcf-3276a7c12485 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:42.979619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eaff9-a3d9-430a-9fcf-3276a7c12485 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:42.979911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eaff9-a3d9-430a-9fcf-3276a7c12485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:42.988425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561410f4-9078-4a00-b150-26e22463a5e3 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:43.000170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c57eaff9-a3d9-430a-9fcf-3276a7c12485 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:43.000170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c57eaff9-a3d9-430a-9fcf-3276a7c12485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:43.012359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:43.053785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:43.071405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 115/221] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:43.072421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50799354-3ccd-49b5-8644-b4e93d36e126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:43.073194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50799354-3ccd-49b5-8644-b4e93d36e126 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.073596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50799354-3ccd-49b5-8644-b4e93d36e126 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.074006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50799354-3ccd-49b5-8644-b4e93d36e126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.074873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50799354-3ccd-49b5-8644-b4e93d36e126 None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=7db7471fa61b4255868f9a7d8217e800) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:43.075310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50799354-3ccd-49b5-8644-b4e93d36e126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:43.076906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50799354-3ccd-49b5-8644-b4e93d36e126 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:43.090930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 107/222] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:42 2022] GET /identity/v3/role_assignments?user.id=7db7471fa61b4255868f9a7d8217e800&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:43.094467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57eaff9-a3d9-430a-9fcf-3276a7c12485 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.156003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58014b5c-1c19-494b-ab27-20b8649c4eef None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.156373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58014b5c-1c19-494b-ab27-20b8649c4eef None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.156625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58014b5c-1c19-494b-ab27-20b8649c4eef None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.158019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 108/223] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:43.163749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ffec-7dea-4bb2-9262-00f10cacb4a6 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.164076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ffec-7dea-4bb2-9262-00f10cacb4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.164352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ffec-7dea-4bb2-9262-00f10cacb4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.196308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c901ffec-7dea-4bb2-9262-00f10cacb4a6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:43.197553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c901ffec-7dea-4bb2-9262-00f10cacb4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:43.264051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57eaff9-a3d9-430a-9fcf-3276a7c12485 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:43.305618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:43.306361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.306643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.306913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.318278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 109/224] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:43.318471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:43.323249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:43.324085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:43.324914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:43.325927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 116/225] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:43.331784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901ffec-7dea-4bb2-9262-00f10cacb4a6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.340493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.381107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:43.463084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7165c-4b51-4a88-bbe9-11d059341b3d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:43.498990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:43.500601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.501102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.501569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.512636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:43.518915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:43.519756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:43.520870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:43.523246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 117/226] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:43.534875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.556492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901ffec-7dea-4bb2-9262-00f10cacb4a6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:43.560366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:43.614349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:43.614969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.615248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.615521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.624236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:43.624610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:43.626549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:43.644633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:43.654040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 110/227] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:43.654915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8824d5-2755-4492-a79d-8479d9f1f8c4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:43.673204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.697565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:43.698292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.698642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.698990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.699726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:43.700097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:43.701978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:43.719344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 118/228] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:43.727213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.831630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6f910-71ba-4499-a065-560a6b4f2a21 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:43.898410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df3689-8ec9-47f7-a575-da1e67eba946 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:43.902151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:43.903003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.903454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.903894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] PATH_INFO: `/v3/users/4b23a40791d54b2fb6531c2fccff996f` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.905359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] RBAC: Authorizing `identity:get_user(user_id=4b23a40791d54b2fb6531c2fccff996f)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:43.907173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:43.913280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 111/229] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/users/4b23a40791d54b2fb6531c2fccff996f => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:43.915301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.972513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:43.973481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:43.973929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:43.974373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] PATH_INFO: `/v3/users/4b23a40791d54b2fb6531c2fccff996f` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:43.975793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] RBAC: Authorizing `identity:get_user(user_id=4b23a40791d54b2fb6531c2fccff996f)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:43.977696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:43.982519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 119/230] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/users/4b23a40791d54b2fb6531c2fccff996f => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:43.990982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:43.999201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:44.036514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:44.122539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824e519-cdeb-4d0d-84db-cd450862d56c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:44.146313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2896093-be84-4fba-9063-d937d261bc51 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:44.185592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:44.186839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.187348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.188586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.189865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:44.191482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:44.198608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 120/231] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:44.200009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:44.200125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:44.200818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.201127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.201381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.204940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:44.206405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:44.208253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 112/232] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:43 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:44.214682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:44.316534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:44.329450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563d92-4fce-4630-9daf-2a0fdc4e2214 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:44.379221 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fea19-5548-412d-9c84-1955001a898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:44.380184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fea19-5548-412d-9c84-1955001a898b None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.380666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fea19-5548-412d-9c84-1955001a898b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.381409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fea19-5548-412d-9c84-1955001a898b None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/4b23a40791d54b2fb6531c2fccff996f/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.384010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fea19-5548-412d-9c84-1955001a898b None admin] RBAC: Authorizing `identity:create_grant(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d, user_id=4b23a40791d54b2fb6531c2fccff996f, role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:44.385846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fea19-5548-412d-9c84-1955001a898b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:44.402823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 121/233] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:44 2022] PUT /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/4b23a40791d54b2fb6531c2fccff996f/roles/491f8de5da624289a3a9b28c155d764c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:41:44.448548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93742b31-0c04-4309-b5e1-fc5199638b41 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:44.463849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:44.511365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b9a2e-429c-41e0-b66a-e6ff5bbd75c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:44.512060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b9a2e-429c-41e0-b66a-e6ff5bbd75c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.512366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b9a2e-429c-41e0-b66a-e6ff5bbd75c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.512644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b9a2e-429c-41e0-b66a-e6ff5bbd75c8 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/4b23a40791d54b2fb6531c2fccff996f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.517605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b9a2e-429c-41e0-b66a-e6ff5bbd75c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d, user_id=4b23a40791d54b2fb6531c2fccff996f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:44.519109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b9a2e-429c-41e0-b66a-e6ff5bbd75c8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:44.538769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 113/234] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:44 2022] PUT /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/4b23a40791d54b2fb6531c2fccff996f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:41:44.808571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f1064-26c3-4cfe-9568-f7934865dd06 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.809081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f1064-26c3-4cfe-9568-f7934865dd06 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.809154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f1064-26c3-4cfe-9568-f7934865dd06 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.810695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 122/235] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:44.816075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f0e34-2119-41e6-90af-2d6454227981 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.816507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f0e34-2119-41e6-90af-2d6454227981 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.816684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f0e34-2119-41e6-90af-2d6454227981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.837689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3c1f0e34-2119-41e6-90af-2d6454227981 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:44.838914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3c1f0e34-2119-41e6-90af-2d6454227981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:44.928908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d20d-3b00-4587-a897-5cdd10eb89e3 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.929220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d20d-3b00-4587-a897-5cdd10eb89e3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.929462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d20d-3b00-4587-a897-5cdd10eb89e3 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.931052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 123/236] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:44.932093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 114/237] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:44.949240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f880d-5b8d-4c06-bcc9-d119d24d6676 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:44.949533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f880d-5b8d-4c06-bcc9-d119d24d6676 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:44.949783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f880d-5b8d-4c06-bcc9-d119d24d6676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:44.975658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f0e34-2119-41e6-90af-2d6454227981 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:44.980543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2b1f880d-5b8d-4c06-bcc9-d119d24d6676 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:44.981810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2b1f880d-5b8d-4c06-bcc9-d119d24d6676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:45.107796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 124/238] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:45.141228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:45.144707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f0e34-2119-41e6-90af-2d6454227981 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:45.171457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f880d-5b8d-4c06-bcc9-d119d24d6676 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:45.197949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:45.198593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:45.198869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:45.199135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:45.203085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:45.205195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:45.208345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 115/239] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:45.218677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:45.316009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:45.358939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f880d-5b8d-4c06-bcc9-d119d24d6676 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:45.393709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbaf93-0bf1-4505-b649-70e87b965f71 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:45.426534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:45.426534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:45.426534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:45.426534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:45.439083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:45.443830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:45.444728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:45.445503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 125/240] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:45.451934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:45.456887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:45.457538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:45.457825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:45.458100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:45.458761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:45.460242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:45.524916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 116/241] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:41:45 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:45.586247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54048b84-00d2-4342-9fec-b7568f0dbeff None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:45.633857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b092547-6dfe-4c02-9312-67e2e3bde901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:45.634862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b092547-6dfe-4c02-9312-67e2e3bde901 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:45.635383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b092547-6dfe-4c02-9312-67e2e3bde901 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:45.635854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b092547-6dfe-4c02-9312-67e2e3bde901 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:45.636805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b092547-6dfe-4c02-9312-67e2e3bde901 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:45.637317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b092547-6dfe-4c02-9312-67e2e3bde901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:45.639259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b092547-6dfe-4c02-9312-67e2e3bde901 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:45.650015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 126/242] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:45.661334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:45.700491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:45.703807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1b64-9bcd-4e4b-a015-c08b4ce4ec94 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:45.704388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1b64-9bcd-4e4b-a015-c08b4ce4ec94 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:45.705021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1b64-9bcd-4e4b-a015-c08b4ce4ec94 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:45.706647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 127/243] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:45.713424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aa233-b8c8-4443-a3f3-d6f60265ec4d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:45.714032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aa233-b8c8-4443-a3f3-d6f60265ec4d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:45.714504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aa233-b8c8-4443-a3f3-d6f60265ec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:45.723340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:45.737684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6d3aa233-b8c8-4443-a3f3-d6f60265ec4d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:45.739364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6d3aa233-b8c8-4443-a3f3-d6f60265ec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:45.803472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369b849-e47d-4572-aaac-14663e855e6a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:45.831463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 128/244] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:45.849516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:45.851259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:45.851259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:45.851259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] PATH_INFO: `/v3/users/2339776e5024441bafb1a9754f009a72` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:45.855032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3aa233-b8c8-4443-a3f3-d6f60265ec4d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:45.869956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] RBAC: Authorizing `identity:get_user(user_id=2339776e5024441bafb1a9754f009a72)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:45.871817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:45.874881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 117/245] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:45 2022] GET /identity/v3/users/2339776e5024441bafb1a9754f009a72 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:45.880983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:46.007307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3aa233-b8c8-4443-a3f3-d6f60265ec4d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.013377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:46.043397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8d73a-a960-4eb0-aec1-2301fcf6880c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.045084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:46.051601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.052259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.052542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.052810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.067896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:46.072992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:46.073837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:46.074596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 129/246] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:46.080678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:46.092521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.093468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.094008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.094483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.167811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:46.169744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:46.194438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d2692-555b-4b36-92a4-7a9b517cd493 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.233487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.234109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.234887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.235470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.235926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:46.236290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:46.237688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:46.247817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 130/247] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:46.253477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:46.257651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:46.281377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 118/248] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:46.289489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:46.326247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:46.380603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c7464-d294-4337-99a5-82721ca0096a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.418882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975e250-2d9a-4176-9237-9ae279c1a16f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.434178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.435333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.435762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.436205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] PATH_INFO: `/v3/users/7db7471fa61b4255868f9a7d8217e800` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.438048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] RBAC: Authorizing `identity:get_user(user_id=7db7471fa61b4255868f9a7d8217e800)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:46.440386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:46.442509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 131/249] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:46 2022] GET /identity/v3/users/7db7471fa61b4255868f9a7d8217e800 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:46.449139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:46.493772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.494591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.494973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.495341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.496622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:46.498164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:46.505424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 119/250] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:46 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:46.511051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:46.571591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ad1b4-757e-479d-b1b7-efcf37ef8973 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.600618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:46.614971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.616206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.616665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.617086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.618648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:46.620275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:46.622385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 132/251] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:46 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:46.623887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:46.632364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:46.726294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacafd5-d808-41f7-84de-54dd7e507321 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.809489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea08b19-fe76-419c-b530-6082dd33b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.810031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea08b19-fe76-419c-b530-6082dd33b1af None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.810317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea08b19-fe76-419c-b530-6082dd33b1af None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.810591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea08b19-fe76-419c-b530-6082dd33b1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.811325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea08b19-fe76-419c-b530-6082dd33b1af None admin] RBAC: Adding query filter params (role.id=13803fbdb99c469b9c329ce4e9d995d2, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=2339776e5024441bafb1a9754f009a72) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:46.811636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea08b19-fe76-419c-b530-6082dd33b1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:46.813886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea08b19-fe76-419c-b530-6082dd33b1af None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:46.830572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 120/252] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:46 2022] GET /identity/v3/role_assignments?user.id=2339776e5024441bafb1a9754f009a72&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=13803fbdb99c469b9c329ce4e9d995d2 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:46.834199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ac3f-b53d-4bee-acc0-c82205786b17 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:46.896966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1ee17-6ee0-42bc-9d16-33f90dac13f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:46.897668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ee17-6ee0-42bc-9d16-33f90dac13f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:46.897955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ee17-6ee0-42bc-9d16-33f90dac13f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:46.898240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ee17-6ee0-42bc-9d16-33f90dac13f0 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/7db7471fa61b4255868f9a7d8217e800/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:46.900327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1ee17-6ee0-42bc-9d16-33f90dac13f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d, user_id=7db7471fa61b4255868f9a7d8217e800, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:46.901811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1ee17-6ee0-42bc-9d16-33f90dac13f0 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:46.921849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 133/253] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:46 2022] PUT /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/7db7471fa61b4255868f9a7d8217e800/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:41:47.042868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7790b-f7fb-4858-a603-2136c6b1ec94 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:47.043403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7790b-f7fb-4858-a603-2136c6b1ec94 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:47.043403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7790b-f7fb-4858-a603-2136c6b1ec94 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:47.045210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 121/254] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:47.051760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec059c6f-12fb-482c-9068-356b41608d36 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:47.052063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec059c6f-12fb-482c-9068-356b41608d36 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:47.052381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec059c6f-12fb-482c-9068-356b41608d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:47.081631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ec059c6f-12fb-482c-9068-356b41608d36 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:47.082823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ec059c6f-12fb-482c-9068-356b41608d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:47.147323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:47.171640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 134/255] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:47.197931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7790b-f7fb-4858-a603-2136c6b1ec94 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:47.222519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54164-5d7e-47a3-a9d2-e4bd8812136f None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:47.223141 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54164-5d7e-47a3-a9d2-e4bd8812136f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:47.223643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54164-5d7e-47a3-a9d2-e4bd8812136f None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:47.225745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 135/256] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:47.239408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30513da9-45a3-44e1-8b2c-7ec3c59d3ebb None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:47.240251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30513da9-45a3-44e1-8b2c-7ec3c59d3ebb None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:47.240734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30513da9-45a3-44e1-8b2c-7ec3c59d3ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:47.273643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-30513da9-45a3-44e1-8b2c-7ec3c59d3ebb None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:47.275181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-30513da9-45a3-44e1-8b2c-7ec3c59d3ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:47.373032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 136/257] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:47.373859 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7790b-f7fb-4858-a603-2136c6b1ec94 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:47.397620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:47.410060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30513da9-45a3-44e1-8b2c-7ec3c59d3ebb None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:47.465411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:47.466068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:47.466348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:47.466619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:47.493351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:47.498146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:47.499018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:47.499775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 122/258] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:41:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:47.513170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:47.586443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30513da9-45a3-44e1-8b2c-7ec3c59d3ebb None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:47.608205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:47.672657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bb686-0c7e-4d23-a2aa-50df0d06785d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:47.686900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:47.687585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:47.687870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:47.688139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:47.713679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:47.718732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:47.720203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:41:47.720983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:47.726692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 137/259] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:47.736806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:47.737186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:47.737431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:47.737712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:47.737981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:47.738651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:47.738955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:47.740364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:47.756027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 123/260] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:41:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:47.764481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:47.823011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:47.963815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005ae46-8f97-4b26-a0d1-65105e3f4089 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:47.992825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fc526-8db5-4110-8671-630c3c24b84a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:48.041550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:48.042180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:48.042460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:48.042736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:48.043413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:48.043785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:48.045240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:48.066348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 138/261] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:48.072196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:48.073623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:48.074346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:48.074547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:48.074818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] PATH_INFO: `/v3/users/4b23a40791d54b2fb6531c2fccff996f` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:48.081141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] RBAC: Authorizing `identity:get_user(user_id=4b23a40791d54b2fb6531c2fccff996f)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:48.082537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:48.084457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 124/262] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:47 2022] GET /identity/v3/users/4b23a40791d54b2fb6531c2fccff996f => generated 283 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:48.091824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:48.135478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:48.242922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:48.246174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64fdda-3a7d-40b9-9729-538b982901bf None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:48.299984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:48.300962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:48.301244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:48.301516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:48.342158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b530f3-158f-4086-ac29-c11d8c41e3ea None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:48.432482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:48.434473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:48.479223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:48.479873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:48.480162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:48.480439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] PATH_INFO: `/v3/users/4b23a40791d54b2fb6531c2fccff996f` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:48.481570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] RBAC: Authorizing `identity:get_user(user_id=4b23a40791d54b2fb6531c2fccff996f)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:48.483056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:48.486996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 139/263] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:48 2022] GET /identity/v3/users/4b23a40791d54b2fb6531c2fccff996f => generated 283 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:48.493205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:48.624594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:48.634903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 125/264] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:41:48.642403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:48.693205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:48.701192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2b558-6322-4f68-ae3d-28bfa6c53f59 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:48.780653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d7b2c-81a1-4a99-a57e-c6249943474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:48.781903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7b2c-81a1-4a99-a57e-c6249943474f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:48.782186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7b2c-81a1-4a99-a57e-c6249943474f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:48.782456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7b2c-81a1-4a99-a57e-c6249943474f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:48.860936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e695f-a329-43e5-9e11-67ffb662bb35 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:48.952857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:48.953121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d7b2c-81a1-4a99-a57e-c6249943474f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:48.953898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:48.953898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:48.954191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:48.954563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d7b2c-81a1-4a99-a57e-c6249943474f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:48.955379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:48.956851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:48.959596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 126/265] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:48 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:48.964618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:49.111071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:49.147238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:49.160838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 140/266] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:49.162204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac1065-bc68-42e4-ab45-24e83e4a914f None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.162528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac1065-bc68-42e4-ab45-24e83e4a914f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.162800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac1065-bc68-42e4-ab45-24e83e4a914f None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.164216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 141/267] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:49.167242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac1065-bc68-42e4-ab45-24e83e4a914f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:49.184351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b968f-1052-4615-ac9c-9a16fb318bc4 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:49.231809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f32887-7c6e-47a3-8dc0-1fff0d9328c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:49.232452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32887-7c6e-47a3-8dc0-1fff0d9328c3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.232739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32887-7c6e-47a3-8dc0-1fff0d9328c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.233017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32887-7c6e-47a3-8dc0-1fff0d9328c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.233752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f32887-7c6e-47a3-8dc0-1fff0d9328c3 None admin] RBAC: Adding query filter params (role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=4b23a40791d54b2fb6531c2fccff996f) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:49.234064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f32887-7c6e-47a3-8dc0-1fff0d9328c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:49.235556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f32887-7c6e-47a3-8dc0-1fff0d9328c3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:49.244547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 127/268] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:48 2022] GET /identity/v3/role_assignments?user.id=4b23a40791d54b2fb6531c2fccff996f&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=491f8de5da624289a3a9b28c155d764c => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:49.246108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca070d-d666-46a7-903d-a6ff358fcd55 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.246531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca070d-d666-46a7-903d-a6ff358fcd55 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.246963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca070d-d666-46a7-903d-a6ff358fcd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.269871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-1fca070d-d666-46a7-903d-a6ff358fcd55 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:49.271386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1fca070d-d666-46a7-903d-a6ff358fcd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:49.359939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 128/269] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:49.361465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e8d0-387d-4318-ad04-922ad57af616 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.361694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e8d0-387d-4318-ad04-922ad57af616 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.361964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e8d0-387d-4318-ad04-922ad57af616 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.363347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 129/270] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:49.369698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c21837-88ad-42ce-9b7b-473e0e670b83 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.370019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c21837-88ad-42ce-9b7b-473e0e670b83 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.370970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c21837-88ad-42ce-9b7b-473e0e670b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.409506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b5c21837-88ad-42ce-9b7b-473e0e670b83 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:49.411052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b5c21837-88ad-42ce-9b7b-473e0e670b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:49.441160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac1065-bc68-42e4-ab45-24e83e4a914f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:49.488602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:49.489289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.489570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.489840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.490958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:49.492459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:49.494414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 142/271] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:49 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:49.495193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:49.507088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:49.528618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 130/272] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:49.530184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c21837-88ad-42ce-9b7b-473e0e670b83 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:49.588274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:49.629825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1e153-749d-4230-93c6-5eb0df6b0883 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:49.667191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c21837-88ad-42ce-9b7b-473e0e670b83 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:49.670779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:49.671424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.671699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.671991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.686623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:49.692865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:49.693764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:49.694531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 143/273] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:49.695167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:49.713790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:49.714418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.714696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.714966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.715872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=4b23a40791d54b2fb6531c2fccff996f) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:49.716104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:49.717656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:49.724541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 131/274] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:49 2022] GET /identity/v3/role_assignments?user.id=4b23a40791d54b2fb6531c2fccff996f&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:49.725667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:49.808293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:49.830662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626471-4a02-4f77-99a3-e256ef8aa526 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:49.860797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:49.869626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d3010-4b1e-4b4f-8cae-681ed3e75b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:49.870272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3010-4b1e-4b4f-8cae-681ed3e75b80 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.870548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3010-4b1e-4b4f-8cae-681ed3e75b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.870817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3010-4b1e-4b4f-8cae-681ed3e75b80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.871119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f24221-1f25-42a8-b1df-da1f23946abf None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:49.871980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d3010-4b1e-4b4f-8cae-681ed3e75b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:49.873478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d3010-4b1e-4b4f-8cae-681ed3e75b80 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:49.879695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 144/275] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:49.879695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ce34b-4c44-41a5-a56b-22efa45b7d13 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.879695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ce34b-4c44-41a5-a56b-22efa45b7d13 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.879695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ce34b-4c44-41a5-a56b-22efa45b7d13 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.879695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 145/276] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:49.880193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be693-aac1-4cd1-95a9-aeb430793b1f None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.880528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be693-aac1-4cd1-95a9-aeb430793b1f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.880637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be693-aac1-4cd1-95a9-aeb430793b1f None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.882050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 146/277] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:49.882668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be693-aac1-4cd1-95a9-aeb430793b1f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:49.926717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd7b60-9c31-4305-a998-f57410f39d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:49.927348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd7b60-9c31-4305-a998-f57410f39d63 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.928448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd7b60-9c31-4305-a998-f57410f39d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.928753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd7b60-9c31-4305-a998-f57410f39d63 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.929460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd7b60-9c31-4305-a998-f57410f39d63 None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:49.929771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd7b60-9c31-4305-a998-f57410f39d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:49.931245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd7b60-9c31-4305-a998-f57410f39d63 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:49.942957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 132/278] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:49.944204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c81d8-008c-4772-b48f-a0850022c07c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:49.944493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c81d8-008c-4772-b48f-a0850022c07c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:49.944767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c81d8-008c-4772-b48f-a0850022c07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:49.964833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-935c81d8-008c-4772-b48f-a0850022c07c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:49.966030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-935c81d8-008c-4772-b48f-a0850022c07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:50.023982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be693-aac1-4cd1-95a9-aeb430793b1f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.038390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 133/279] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:50.041404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba476f-64f8-48d1-a9dd-b6352546e8ac None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.041692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba476f-64f8-48d1-a9dd-b6352546e8ac None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.041969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba476f-64f8-48d1-a9dd-b6352546e8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.059531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-bbba476f-64f8-48d1-a9dd-b6352546e8ac None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:50.060825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bbba476f-64f8-48d1-a9dd-b6352546e8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:50.065393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.066001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.066281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.066583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.067193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:50.068688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:50.089757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 147/280] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:41:49 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:50.090576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.124830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:50.161050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 134/281] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:50.161050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba476f-64f8-48d1-a9dd-b6352546e8ac None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.162632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:50.219429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886792e0-9b3c-4454-a859-c0d23e07eaf6 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.272208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.272208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.272208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.272208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] PATH_INFO: `/v3/users/2cd403140ab74bc7a6587c2bfd8a9f74` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.282400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba476f-64f8-48d1-a9dd-b6352546e8ac None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.291788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd403140ab74bc7a6587c2bfd8a9f74)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:50.293397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:50.295097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 148/282] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/users/2cd403140ab74bc7a6587c2bfd8a9f74 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:50.296874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.335347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.335986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.336342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.336612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.348239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:50.353265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:50.354138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:50.354896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:50.356225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 135/283] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:50.357249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.412074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:50.443833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140c421-5dbe-4047-be85-31bcde98af28 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.458592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:50.490975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.492102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.492519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.492930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.497404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262ca8d-f0ce-47d4-90d6-9bd590d82763 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.508902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:50.515157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:50.516595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:50.517990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 149/284] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:50.518781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.543616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.544254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.544535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.544806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.545551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, user.id=2cd403140ab74bc7a6587c2bfd8a9f74, scope.system=all) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:50.545831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:50.547930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:50.554377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 136/285] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/role_assignments?user.id=2cd403140ab74bc7a6587c2bfd8a9f74&scope.system=all&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:50.555296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.683628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c004-ca11-4451-ac0e-f1d2d132bdc1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.686537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:50.706169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20fe28-1cf8-4e81-be6d-6c2454067770 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.733312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.734311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.734868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.735362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.736296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:50.736816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:50.739320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:50.749227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 137/286] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:50.757984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.787483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.788098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.788425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.788696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.789368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:50.789670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:50.791599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:50.814969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 150/287] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:50.820533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:50.876548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:50.918180 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af3354-2a1e-4518-9745-e9595ede2180 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:50.991563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:50.992134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:50.992436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:50.992711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] PATH_INFO: `/v3/users/2339776e5024441bafb1a9754f009a72` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:50.994831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] RBAC: Authorizing `identity:get_user(user_id=2339776e5024441bafb1a9754f009a72)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:50.996335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:50.997935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 138/288] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/users/2339776e5024441bafb1a9754f009a72 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:51.006527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:51.018809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b15d-b0f8-47a0-a5d5-963a2d4c59c9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:51.054171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:51.054818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:51.055081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:51.055349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] PATH_INFO: `/v3/users/7db7471fa61b4255868f9a7d8217e800` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:51.056573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=7db7471fa61b4255868f9a7d8217e800)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:51.058168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:51.060163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 151/289] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:50 2022] GET /identity/v3/users/7db7471fa61b4255868f9a7d8217e800 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:51.065031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:51.066152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:51.191943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c8744-ed59-425b-a59d-7554435ab9b2 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:51.200285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86abec-d345-468e-b2fe-cda636c22bb8 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:51.200691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:51.237848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:51.238455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:51.238740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:51.239001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:51.262218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:51.262787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:51.263065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:51.263333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:51.264513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:51.267677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:51.269853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 139/290] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:51 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:51.281877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:51.400007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:51.401507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:51.445356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:51.450094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd003f74-3a3d-41ee-895a-9689eadae06a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:51.512967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:51.513643 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:51.513956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:51.514267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/2339776e5024441bafb1a9754f009a72/roles/13803fbdb99c469b9c329ce4e9d995d2` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:51.516719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] RBAC: Authorizing `identity:create_grant(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d, user_id=2339776e5024441bafb1a9754f009a72, role_id=13803fbdb99c469b9c329ce4e9d995d2)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:51.518734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:51.536367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 140/291] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:41:51 2022] PUT /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/users/2339776e5024441bafb1a9754f009a72/roles/13803fbdb99c469b9c329ce4e9d995d2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:41:51.597480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:51.601738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 152/292] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:51.607484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:51.738720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2719-352d-4776-8885-39cd7416fddb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:51.775196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614742ec-2939-46e9-9a0b-159d5376e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:51.775758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614742ec-2939-46e9-9a0b-159d5376e69d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:51.776036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614742ec-2939-46e9-9a0b-159d5376e69d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:51.776335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614742ec-2939-46e9-9a0b-159d5376e69d None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:51.777454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614742ec-2939-46e9-9a0b-159d5376e69d None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:51.778884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614742ec-2939-46e9-9a0b-159d5376e69d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:51.780821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 141/293] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:51 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:51.787080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:51.896024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdf9d-898a-49a8-a4ae-7c686b616aa4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:51.933312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f354669-be8d-425b-b26d-8c8aa7ccb0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:51.934019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f354669-be8d-425b-b26d-8c8aa7ccb0cc None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:51.934300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f354669-be8d-425b-b26d-8c8aa7ccb0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:51.934571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f354669-be8d-425b-b26d-8c8aa7ccb0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:51.935315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f354669-be8d-425b-b26d-8c8aa7ccb0cc None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=7db7471fa61b4255868f9a7d8217e800) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:51.935625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f354669-be8d-425b-b26d-8c8aa7ccb0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:51.937146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f354669-be8d-425b-b26d-8c8aa7ccb0cc None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:51.944399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 153/294] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:51 2022] GET /identity/v3/role_assignments?user.id=7db7471fa61b4255868f9a7d8217e800&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:52.589728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e455be-22df-4d3e-90d4-8e05d8c3f380 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:52.590246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e455be-22df-4d3e-90d4-8e05d8c3f380 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:52.590349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e455be-22df-4d3e-90d4-8e05d8c3f380 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:52.592032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 142/295] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:52.598598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18e1c-8a22-44a1-b6ed-feba8480f7c7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:52.599055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18e1c-8a22-44a1-b6ed-feba8480f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:52.599126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18e1c-8a22-44a1-b6ed-feba8480f7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:52.618655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c6a18e1c-8a22-44a1-b6ed-feba8480f7c7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:52.619837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c6a18e1c-8a22-44a1-b6ed-feba8480f7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:52.684058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:52.695198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969cfae-abc3-4280-8ff7-0d073d768787 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:52.695488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969cfae-abc3-4280-8ff7-0d073d768787 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:52.695748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969cfae-abc3-4280-8ff7-0d073d768787 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:52.697091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 143/296] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:52.697352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 154/297] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:52.705923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61befe7-81d7-449e-acec-eb105b23589f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:52.706255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61befe7-81d7-449e-acec-eb105b23589f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:52.706567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61befe7-81d7-449e-acec-eb105b23589f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:52.719675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a18e1c-8a22-44a1-b6ed-feba8480f7c7 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:52.732935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b61befe7-81d7-449e-acec-eb105b23589f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:52.735115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b61befe7-81d7-449e-acec-eb105b23589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:52.764514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:52.825369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 144/298] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:52.843073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a18e1c-8a22-44a1-b6ed-feba8480f7c7 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:52.846349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61befe7-81d7-449e-acec-eb105b23589f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:52.884886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:52.885528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:52.885807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:52.888517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:52.889846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:52.892571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:52.895322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 155/299] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:52.900245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:52.961948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61befe7-81d7-449e-acec-eb105b23589f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:52.971575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:53.003276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0070020-f1aa-4a6a-8d29-9543bed63542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.004336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0070020-f1aa-4a6a-8d29-9543bed63542 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.004796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0070020-f1aa-4a6a-8d29-9543bed63542 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.005145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0070020-f1aa-4a6a-8d29-9543bed63542 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.012608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e8b76-1816-46a7-b6fc-9353708f8cfe None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.016642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0070020-f1aa-4a6a-8d29-9543bed63542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-e0070020-f1aa-4a6a-8d29-9543bed63542 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:53.021530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:53.022397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:53.023160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 145/300] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:53.024379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6049a-c2b4-44b6-b03a-d760fce95bee None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.024379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6049a-c2b4-44b6-b03a-d760fce95bee None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.024498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6049a-c2b4-44b6-b03a-d760fce95bee None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.028193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 146/301] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:53.029425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6049a-c2b4-44b6-b03a-d760fce95bee None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:53.050532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:53.056001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f555f9-7d5c-4900-bdf0-ddeaef80aefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.056657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f555f9-7d5c-4900-bdf0-ddeaef80aefd None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.056936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f555f9-7d5c-4900-bdf0-ddeaef80aefd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.057206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f555f9-7d5c-4900-bdf0-ddeaef80aefd None admin] PATH_INFO: `/v3/groups` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.057831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f555f9-7d5c-4900-bdf0-ddeaef80aefd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:53.059263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f555f9-7d5c-4900-bdf0-ddeaef80aefd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:53.076714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 156/302] 10.222.0.25 () {60 vars in 1134 bytes} [Wed Aug 10 16:41:52 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:41:53.084390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48903-557c-4aea-b9b6-bd82d6cc42a2 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.084390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48903-557c-4aea-b9b6-bd82d6cc42a2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.084390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48903-557c-4aea-b9b6-bd82d6cc42a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.108650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-81a48903-557c-4aea-b9b6-bd82d6cc42a2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:53.110723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-81a48903-557c-4aea-b9b6-bd82d6cc42a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:53.184210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6049a-c2b4-44b6-b03a-d760fce95bee None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.204771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 157/303] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:53.226851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a48903-557c-4aea-b9b6-bd82d6cc42a2 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:53.233891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.234521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.235338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.235614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.236950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:53.237260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:53.239291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:53.255312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 147/304] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:53.262691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:53.297148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:53.358977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a48903-557c-4aea-b9b6-bd82d6cc42a2 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.375663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:53.401784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86aab5-c2a5-46fc-85ee-db4ede44fbc8 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.404215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.404978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.405404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.405785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.418005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:53.423871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:53.424953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:53.425740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:53.426675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 158/305] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:53.436200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:53.447472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.448126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.448442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.448722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] PATH_INFO: `/v3/users/dbafb6fce42048bea91e400e5059ab71` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.467369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] RBAC: Authorizing `identity:get_user(user_id=dbafb6fce42048bea91e400e5059ab71)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:53.469610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:53.472410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 148/306] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:53 2022] GET /identity/v3/users/dbafb6fce42048bea91e400e5059ab71 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:53.478205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:53.542423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b5060-6420-4795-a44a-bb0cdef79875 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.558517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:53.581262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.581876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.582199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.582473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.583143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:53.583440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:53.584979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:53.586559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b07d92-5de7-4d93-b9e4-9faa7bf5d8df None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.599180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 159/307] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:53.600470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:53.624103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.624754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.625011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.625710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.626483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] RBAC: Adding query filter params (role.id=13803fbdb99c469b9c329ce4e9d995d2, user.id=dbafb6fce42048bea91e400e5059ab71, scope.system=all) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:53.626793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:53.628244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:53.633549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 149/308] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 10 16:41:53 2022] GET /identity/v3/role_assignments?user.id=dbafb6fce42048bea91e400e5059ab71&scope.system=all&role.id=13803fbdb99c469b9c329ce4e9d995d2 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:53.724491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a1a29-3ccb-435a-b148-bec96d1fc507 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.763484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62195579-7067-4c1d-aa2b-d118cd59c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.764050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62195579-7067-4c1d-aa2b-d118cd59c806 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.764357 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62195579-7067-4c1d-aa2b-d118cd59c806 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.764607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62195579-7067-4c1d-aa2b-d118cd59c806 None admin] PATH_INFO: `/v3/users/2cd403140ab74bc7a6587c2bfd8a9f74` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.766326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62195579-7067-4c1d-aa2b-d118cd59c806 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd403140ab74bc7a6587c2bfd8a9f74)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:53.767909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62195579-7067-4c1d-aa2b-d118cd59c806 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:53.774393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 160/309] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:53 2022] GET /identity/v3/users/2cd403140ab74bc7a6587c2bfd8a9f74 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:53.775845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:53.779133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:53.882307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffa5bf-dc03-4456-8d5b-bc598f35084f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:53.911653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d500-213c-4199-8f83-eee80338c5d4 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.911971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d500-213c-4199-8f83-eee80338c5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.912228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d500-213c-4199-8f83-eee80338c5d4 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.913858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 161/310] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:53.919686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15805c7-0a03-4661-b443-41e6879dad69 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.920005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15805c7-0a03-4661-b443-41e6879dad69 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.920263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15805c7-0a03-4661-b443-41e6879dad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.932802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:53.933782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:53.934276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:53.935261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] PATH_INFO: `/v3/system/users/2cd403140ab74bc7a6587c2bfd8a9f74/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:53.937223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2cd403140ab74bc7a6587c2bfd8a9f74, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:53.938823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:53.941741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a15805c7-0a03-4661-b443-41e6879dad69 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:53.942940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a15805c7-0a03-4661-b443-41e6879dad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:53.956814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 150/311] 10.222.0.25 () {58 vars in 1334 bytes} [Wed Aug 10 16:41:53 2022] PUT /identity/v3/system/users/2cd403140ab74bc7a6587c2bfd8a9f74/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:41:53.991751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:54.040513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 162/312] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:54.057548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:54.138628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:54.177279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c1a7-b911-470e-9adc-59f1fafd3d0e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:54.218536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:54.219563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:54.220042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:54.220568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:54.232125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:54.237870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:54.238745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:54.239518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:54.241870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 151/313] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:54.248850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15805c7-0a03-4661-b443-41e6879dad69 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:54.363959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15805c7-0a03-4661-b443-41e6879dad69 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:54.400591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:54.401202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:54.401481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:54.401753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:54.402418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:54.402716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:54.404101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:54.418066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 163/314] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:54.419430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:54.526185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb33ed-fe60-46ad-bcc7-9eaa8d3aaa9b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:54.552890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:54.564857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:54.565431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:54.566018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:54.566292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] PATH_INFO: `/v3/users/2339776e5024441bafb1a9754f009a72` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:54.567453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] RBAC: Authorizing `identity:get_user(user_id=2339776e5024441bafb1a9754f009a72)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:54.568912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:54.575043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 152/315] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:54 2022] GET /identity/v3/users/2339776e5024441bafb1a9754f009a72 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:54.576456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:54.613850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:54.683311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c36d5-d5dc-4d88-8fe0-6c5b73790484 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:54.720133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:54.720778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:54.721056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:54.721330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:54.793128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:54.794615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:54.830179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:54.899329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 164/316] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:41:54.905227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:55.011207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954b0a1-0636-4e48-84dc-4a9c8e295685 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:55.045981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391aad2-047b-44bc-9b4d-ae683ce4688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:55.046927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391aad2-047b-44bc-9b4d-ae683ce4688f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.047429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391aad2-047b-44bc-9b4d-ae683ce4688f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.047937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391aad2-047b-44bc-9b4d-ae683ce4688f None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.049364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391aad2-047b-44bc-9b4d-ae683ce4688f None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:55.050991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391aad2-047b-44bc-9b4d-ae683ce4688f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:55.053157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 153/317] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:41:54 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:55.059305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:55.168095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd12a3-3814-41cc-b633-37606df976b5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:55.202424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519893fd-aea7-4d54-a87c-51e33c3c6283 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.202740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519893fd-aea7-4d54-a87c-51e33c3c6283 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.202984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519893fd-aea7-4d54-a87c-51e33c3c6283 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.204627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 154/318] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:55.207297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:55.207903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.208201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.208475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.209202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] RBAC: Adding query filter params (role.id=13803fbdb99c469b9c329ce4e9d995d2, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d, user.id=2339776e5024441bafb1a9754f009a72) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:55.209505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:55.209723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d13d5a-fb3d-4225-984a-a4853f0bd9d9 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.210018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d13d5a-fb3d-4225-984a-a4853f0bd9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.210298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d13d5a-fb3d-4225-984a-a4853f0bd9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.210944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:55.212808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:55.218361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 165/319] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:41:55 2022] GET /identity/v3/role_assignments?user.id=2339776e5024441bafb1a9754f009a72&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=13803fbdb99c469b9c329ce4e9d995d2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:55.229635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a2d13d5a-fb3d-4225-984a-a4853f0bd9d9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:55.231233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a2d13d5a-fb3d-4225-984a-a4853f0bd9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:55.304628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:55.314687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 155/320] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:55.336829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:55.448542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf05da-fe1b-458d-afa1-37ee6525a4e3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:55.485535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b8159-bfa2-4b59-92d0-dca9f9382437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:55.486474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b8159-bfa2-4b59-92d0-dca9f9382437 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.486958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b8159-bfa2-4b59-92d0-dca9f9382437 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.487406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b8159-bfa2-4b59-92d0-dca9f9382437 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.488740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b8159-bfa2-4b59-92d0-dca9f9382437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:55.490381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b8159-bfa2-4b59-92d0-dca9f9382437 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:55.492541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 166/321] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:41:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:55.498610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d13d5a-fb3d-4225-984a-a4853f0bd9d9 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:55.605600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d13d5a-fb3d-4225-984a-a4853f0bd9d9 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:55.639429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dce2e-9415-44ca-b957-bf273f71abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:55.640040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dce2e-9415-44ca-b957-bf273f71abd5 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.640349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dce2e-9415-44ca-b957-bf273f71abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.640613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dce2e-9415-44ca-b957-bf273f71abd5 None admin] PATH_INFO: `/v3/groups` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.641939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dce2e-9415-44ca-b957-bf273f71abd5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:55.643364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dce2e-9415-44ca-b957-bf273f71abd5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:55.659684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 156/322] 10.222.0.25 () {60 vars in 1134 bytes} [Wed Aug 10 16:41:55 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:41:55.735026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b05c5-9a5d-4bdc-8a7e-88fcb8954143 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.735581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b05c5-9a5d-4bdc-8a7e-88fcb8954143 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.735581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b05c5-9a5d-4bdc-8a7e-88fcb8954143 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.737166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 167/323] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:55.742879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41a85-2d02-492b-9160-f7ffdde42709 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:55.743252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41a85-2d02-492b-9160-f7ffdde42709 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:55.743476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41a85-2d02-492b-9160-f7ffdde42709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:55.763755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-22b41a85-2d02-492b-9160-f7ffdde42709 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:55.764980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-22b41a85-2d02-492b-9160-f7ffdde42709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:55.818793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:55.840818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 157/324] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:55.862641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b05c5-9a5d-4bdc-8a7e-88fcb8954143 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:55.941177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:55.990248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b05c5-9a5d-4bdc-8a7e-88fcb8954143 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:56.027721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b527ba-c3bd-4946-99d4-a23cb31d140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:56.028394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b527ba-c3bd-4946-99d4-a23cb31d140e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.028677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b527ba-c3bd-4946-99d4-a23cb31d140e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.028954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b527ba-c3bd-4946-99d4-a23cb31d140e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.039881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b527ba-c3bd-4946-99d4-a23cb31d140e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-35b527ba-c3bd-4946-99d4-a23cb31d140e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:56.045262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:56.046120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:56.046877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 168/325] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:56.052887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b41a85-2d02-492b-9160-f7ffdde42709 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:56.118692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0b73-7e94-4d01-8f9a-b3facb861ad8 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.118982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0b73-7e94-4d01-8f9a-b3facb861ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.119271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0b73-7e94-4d01-8f9a-b3facb861ad8 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.120812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 169/326] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:41:56.126744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865b669-99a6-4359-81bc-e2b0585b79ee None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.127039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865b669-99a6-4359-81bc-e2b0585b79ee None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.127316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865b669-99a6-4359-81bc-e2b0585b79ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.149585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8865b669-99a6-4359-81bc-e2b0585b79ee None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:56.151228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8865b669-99a6-4359-81bc-e2b0585b79ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:56.175693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b41a85-2d02-492b-9160-f7ffdde42709 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:56.221058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:56.221678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.221958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.222229 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.223009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:56.223319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:56.224875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:56.234605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 158/327] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:56.245865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:56.246171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 170/328] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:41:56.269525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865b669-99a6-4359-81bc-e2b0585b79ee None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:56.305175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:56.337911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:56.379578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac7aad-17cd-48b1-ba86-f0fb0f43e1e1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:56.393609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865b669-99a6-4359-81bc-e2b0585b79ee None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:56.418911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:56.419557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.419838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.420114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] PATH_INFO: `/v3/users/dbafb6fce42048bea91e400e5059ab71` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.421303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] RBAC: Authorizing `identity:get_user(user_id=dbafb6fce42048bea91e400e5059ab71)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:56.423396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:56.426399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 159/329] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:56 2022] GET /identity/v3/users/dbafb6fce42048bea91e400e5059ab71 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:56.436493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:56.443445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:56.444086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.444401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.444675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.457059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:56.462144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:56.463002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:56.464342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 171/330] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:56.469828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:56.566113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107207-97f5-4363-8353-7993eca9fa80 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:56.590347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:56.613983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59257b41-7508-4ec0-a331-b2ac3a0356f1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:56.618849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:56.619604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.619973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.620351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] PATH_INFO: `/v3/system/users/dbafb6fce42048bea91e400e5059ab71/roles/13803fbdb99c469b9c329ce4e9d995d2` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.621946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dbafb6fce42048bea91e400e5059ab71, role_id=13803fbdb99c469b9c329ce4e9d995d2)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:56.623513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:56.635989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 160/331] 10.222.0.25 () {58 vars in 1334 bytes} [Wed Aug 10 16:41:56 2022] PUT /identity/v3/system/users/dbafb6fce42048bea91e400e5059ab71/roles/13803fbdb99c469b9c329ce4e9d995d2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:41:56.640173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:56.666000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:56.666745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.667063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.667356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.669135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:56.669466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:56.671092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:56.693013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 172/332] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:41:56.699699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:56.829088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e8958-1553-4230-b396-c3ce645bdd14 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:56.866368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c481ec6-da97-4ae8-aa63-cf20eb6b3e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:56.867161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c481ec6-da97-4ae8-aa63-cf20eb6b3e25 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:56.867502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c481ec6-da97-4ae8-aa63-cf20eb6b3e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:56.867784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c481ec6-da97-4ae8-aa63-cf20eb6b3e25 None admin] PATH_INFO: `/v3/users/2cd403140ab74bc7a6587c2bfd8a9f74` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:56.868985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c481ec6-da97-4ae8-aa63-cf20eb6b3e25 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd403140ab74bc7a6587c2bfd8a9f74)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:56.870507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c481ec6-da97-4ae8-aa63-cf20eb6b3e25 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:56.872553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 161/333] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:41:56 2022] GET /identity/v3/users/2cd403140ab74bc7a6587c2bfd8a9f74 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:56.878803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:57.021872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe62555-46d6-404d-81f1-3efc499882ed None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:57.066431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01620bb-95d2-4232-84ee-3fb0163400ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:57.067052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01620bb-95d2-4232-84ee-3fb0163400ba None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:57.067336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01620bb-95d2-4232-84ee-3fb0163400ba None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:57.067612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01620bb-95d2-4232-84ee-3fb0163400ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:57.068371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01620bb-95d2-4232-84ee-3fb0163400ba None admin] RBAC: Adding query filter params (role.id=230b13c1aa214ed4bb560f74e036a7b7, user.id=2cd403140ab74bc7a6587c2bfd8a9f74, scope.system=all) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:57.068689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01620bb-95d2-4232-84ee-3fb0163400ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:57.070359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01620bb-95d2-4232-84ee-3fb0163400ba None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:57.091200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 173/334] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 10 16:41:56 2022] GET /identity/v3/role_assignments?user.id=2cd403140ab74bc7a6587c2bfd8a9f74&scope.system=all&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 510 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:58.700444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b0148-8030-4f9d-b297-00a30155f387 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:58.701343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b0148-8030-4f9d-b297-00a30155f387 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:58.701811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b0148-8030-4f9d-b297-00a30155f387 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:58.704227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 162/335] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:58.712766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803a307-b188-4344-8cc2-98d32d214d6d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:58.712766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803a307-b188-4344-8cc2-98d32d214d6d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:58.712766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803a307-b188-4344-8cc2-98d32d214d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:58.741255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4803a307-b188-4344-8cc2-98d32d214d6d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:58.742826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4803a307-b188-4344-8cc2-98d32d214d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:58.777282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:58.829579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 174/336] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:58.849322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38caf-5b73-4f13-a5b8-2ab7fd8f8e3a None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:58.849610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38caf-5b73-4f13-a5b8-2ab7fd8f8e3a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:58.849873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38caf-5b73-4f13-a5b8-2ab7fd8f8e3a None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:58.851505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 163/337] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:58.856847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be299a-5df3-4338-b380-fabfd0bab720 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:58.857132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be299a-5df3-4338-b380-fabfd0bab720 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:58.857410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be299a-5df3-4338-b380-fabfd0bab720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:58.866851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f38caf-5b73-4f13-a5b8-2ab7fd8f8e3a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:58.886041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-48be299a-5df3-4338-b380-fabfd0bab720 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:58.887230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-48be299a-5df3-4338-b380-fabfd0bab720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:58.956460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:59.010570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 175/338] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:59.052962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f38caf-5b73-4f13-a5b8-2ab7fd8f8e3a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:59.056187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be299a-5df3-4338-b380-fabfd0bab720 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:59.111826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:59.112515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.112798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.113073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.141702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:59.146984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:59.147856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:41:59.148671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 164/339] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:41:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:41:59.156103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:59.282755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:59.295403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be299a-5df3-4338-b380-fabfd0bab720 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:59.336677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc008b5-dbea-4637-a65f-c42e48511209 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:59.348740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:59.392895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:59.393529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.393796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.394067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.394743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:59.395044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:59.396460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:59.400902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:59.401561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.401842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.402118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.405794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 165/340] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:59.411754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:59.434584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:41:59.440085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:41:59.445129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:41:59.445934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:41:59.451261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 176/341] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:41:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:41:59.454292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:59.595516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94be259-3cf7-49b7-b618-a3323ed8e8cc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:59.642554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a00ae-7af8-40b9-a24c-9c1ac408e56c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:59.644431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e9c71-9cdf-421b-a13e-e25589d9c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:59.645055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9c71-9cdf-421b-a13e-e25589d9c331 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.645332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9c71-9cdf-421b-a13e-e25589d9c331 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.645600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9c71-9cdf-421b-a13e-e25589d9c331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.687036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:59.696688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56e9df-1d77-4c92-bf9b-6d245c55d9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:59.698100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56e9df-1d77-4c92-bf9b-6d245c55d9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.698599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56e9df-1d77-4c92-bf9b-6d245c55d9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.699497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56e9df-1d77-4c92-bf9b-6d245c55d9d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.700685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56e9df-1d77-4c92-bf9b-6d245c55d9d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:41:59.701184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56e9df-1d77-4c92-bf9b-6d245c55d9d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:59.704185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56e9df-1d77-4c92-bf9b-6d245c55d9d8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:59.716566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 177/342] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:41:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:41:59.718738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50e8a6-7faa-44f0-85e2-ce770fcce3a6 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.719814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50e8a6-7faa-44f0-85e2-ce770fcce3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.720393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50e8a6-7faa-44f0-85e2-ce770fcce3a6 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.723020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 178/343] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:59.729362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22f0c-0956-442e-999b-ae7019a04fb1 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.729903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22f0c-0956-442e-999b-ae7019a04fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.730618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22f0c-0956-442e-999b-ae7019a04fb1 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.732432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 179/344] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:41:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:41:59.733663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22f0c-0956-442e-999b-ae7019a04fb1 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:41:59.736797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e9c71-9cdf-421b-a13e-e25589d9c331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:41:59.738341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e9c71-9cdf-421b-a13e-e25589d9c331 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:41:59.760447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:59.868974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 166/345] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:41:59.870169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115271d1-08cf-40d5-8b95-d93004499e61 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.870464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115271d1-08cf-40d5-8b95-d93004499e61 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.870740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115271d1-08cf-40d5-8b95-d93004499e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.888709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22f0c-0956-442e-999b-ae7019a04fb1 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:41:59.890812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-115271d1-08cf-40d5-8b95-d93004499e61 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:41:59.892710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-115271d1-08cf-40d5-8b95-d93004499e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:41:59.939153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:41:59.940063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.940377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.940651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:41:59.971629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:41:59.989220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 167/346] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:41:59.990507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd29c0-4dc3-4e0b-95ac-bfcfa9190c0b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:41:59.990798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd29c0-4dc3-4e0b-95ac-bfcfa9190c0b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:41:59.991073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd29c0-4dc3-4e0b-95ac-bfcfa9190c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.025192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-06dd29c0-4dc3-4e0b-95ac-bfcfa9190c0b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:00.026983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-06dd29c0-4dc3-4e0b-95ac-bfcfa9190c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:00.050018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:00.051515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:00.077627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:00.120065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 168/347] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:00.120906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd29c0-4dc3-4e0b-95ac-bfcfa9190c0b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.180993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 180/348] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:00.182087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.237971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd29c0-4dc3-4e0b-95ac-bfcfa9190c0b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:00.274929 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:00.290201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f71a06-497f-409b-86e9-18eb862f398d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:00.290539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71a06-497f-409b-86e9-18eb862f398d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.291263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71a06-497f-409b-86e9-18eb862f398d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.291263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71a06-497f-409b-86e9-18eb862f398d None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.292342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f71a06-497f-409b-86e9-18eb862f398d None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:00.293902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f71a06-497f-409b-86e9-18eb862f398d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:00.300035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 169/349] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:00.300035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a8525-504d-4c9c-b7b3-c72c681e43df None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.300035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a8525-504d-4c9c-b7b3-c72c681e43df None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.300035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a8525-504d-4c9c-b7b3-c72c681e43df None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.300035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 170/350] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:00.300035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a8525-504d-4c9c-b7b3-c72c681e43df None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.311256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a7f4a-e15a-4c3c-960c-1cf70f3978a6 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:00.334484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:00.360344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:00.361154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.361435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.361708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.376253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:00.383320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:00.384356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:00.385167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 181/351] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:00.385857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.438577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a8525-504d-4c9c-b7b3-c72c681e43df None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:00.486878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:00.487530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.487868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.488163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.502098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:00.507396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:00.508340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:00.509530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 171/352] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:00.510205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.527503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ddc5-8407-45ef-802d-d06cdd2a5913 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:00.579387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d62d9-fffc-405a-9e4a-bc04cd925f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:00.580029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d62d9-fffc-405a-9e4a-bc04cd925f7c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.580349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d62d9-fffc-405a-9e4a-bc04cd925f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.580621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d62d9-fffc-405a-9e4a-bc04cd925f7c None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.583157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d62d9-fffc-405a-9e4a-bc04cd925f7c None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:00.584813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d62d9-fffc-405a-9e4a-bc04cd925f7c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:00.586283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:00.586798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 182/353] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:00.587987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620b68-54f0-462d-9cc7-b25f6ebd4b78 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.588292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620b68-54f0-462d-9cc7-b25f6ebd4b78 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.588570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620b68-54f0-462d-9cc7-b25f6ebd4b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.606676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-df620b68-54f0-462d-9cc7-b25f6ebd4b78 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:00.607883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-df620b68-54f0-462d-9cc7-b25f6ebd4b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:00.644133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a53187-0d96-4a10-9edf-ef929e1fa82c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:00.669025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:00.699754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:00.700457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.700735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.701006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.707208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 183/354] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:00.708064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620b68-54f0-462d-9cc7-b25f6ebd4b78 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.713871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:00.716638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:00.718702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 172/355] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:00.720726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.851502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620b68-54f0-462d-9cc7-b25f6ebd4b78 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:00.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a08a6-922f-4d5b-b49c-2033803d242b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:00.879268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:00.892783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:00.893557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.894193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.894418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.895293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:00.895540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:00.896969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:00.898508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:00.899121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:00.899404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:00.899683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:00.900375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:00.900678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:00.902040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:00.906484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 184/356] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:00.907728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.913826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 173/357] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:00.914690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:00.962301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:01.037379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc42ac5-4264-4c0e-92b9-b131e47697c2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.037730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44e47d-d22d-4aff-ad02-31aa5e404168 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.077507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.078316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.079282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.079282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.080487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:01.082035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:01.083716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 185/358] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:01.084399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:01.089094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e86e0-d630-4522-83cc-6dd69410cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.089721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e86e0-d630-4522-83cc-6dd69410cd09 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.090003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e86e0-d630-4522-83cc-6dd69410cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.090272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e86e0-d630-4522-83cc-6dd69410cd09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.091015 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e86e0-d630-4522-83cc-6dd69410cd09 None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:01.091322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e86e0-d630-4522-83cc-6dd69410cd09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:01.093001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e86e0-d630-4522-83cc-6dd69410cd09 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:01.106740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 174/359] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:00 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:01.108674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4cbcf-03b6-431a-8d65-3580009045e6 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.109205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4cbcf-03b6-431a-8d65-3580009045e6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.109716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4cbcf-03b6-431a-8d65-3580009045e6 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.111976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 175/360] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:01.112658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4cbcf-03b6-431a-8d65-3580009045e6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:01.180027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:01.237051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117756f-031e-48e7-85bf-8acb02fe2c24 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.246233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:01.273054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4cbcf-03b6-431a-8d65-3580009045e6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.285942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.287011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.287011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.287187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.299195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:01.304520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:01.305398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:01.306249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 186/361] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:01.306913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:01.333099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9bf9-d1a1-4043-b17b-fcf5e3e2be07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.333729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9bf9-d1a1-4043-b17b-fcf5e3e2be07 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.334025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9bf9-d1a1-4043-b17b-fcf5e3e2be07 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.334280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9bf9-d1a1-4043-b17b-fcf5e3e2be07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.426805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9bf9-d1a1-4043-b17b-fcf5e3e2be07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:01.426805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9bf9-d1a1-4043-b17b-fcf5e3e2be07 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:01.444300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0bea8-2499-4365-bbef-98c632799a56 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.483569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:01.485708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.487214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.487214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.487538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] PATH_INFO: `/v3/users/dbafb6fce42048bea91e400e5059ab71` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.488747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] RBAC: Authorizing `identity:get_user(user_id=dbafb6fce42048bea91e400e5059ab71)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:01.490206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:01.491980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 187/362] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/users/dbafb6fce42048bea91e400e5059ab71 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:01.492614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:01.540162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:01.565802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 176/363] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:01.569303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ca093-bfee-4618-b674-870b40534fea None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.569861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ca093-bfee-4618-b674-870b40534fea None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.570332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ca093-bfee-4618-b674-870b40534fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.591758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a45ca093-bfee-4618-b674-870b40534fea None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:01.592977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a45ca093-bfee-4618-b674-870b40534fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:01.628726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da68046-de10-4936-b5c9-ac8a0ee40d72 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.674975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.675954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.676475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.676950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.677905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:01.678217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:01.679675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:01.682720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 177/364] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:01.683938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ca093-bfee-4618-b674-870b40534fea None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:01.692242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 188/365] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=491f8de5da624289a3a9b28c155d764c => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:01.692242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:01.802094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:01.825156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:01.830105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2bd9c-8ab4-481b-8fe7-79ec1c0e1726 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.834023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ca093-bfee-4618-b674-870b40534fea None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:01.872298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.873483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.873931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.874354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.875233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] RBAC: Adding query filter params (role.id=13803fbdb99c469b9c329ce4e9d995d2, user.id=dbafb6fce42048bea91e400e5059ab71, scope.system=all) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:01.875768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:01.877356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:01.877986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:01.878164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:01.878289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:01.878520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:01.879185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:01.879488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:01.880886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:01.889967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 178/366] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:01.890621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:01.898562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 189/367] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/role_assignments?user.id=dbafb6fce42048bea91e400e5059ab71&scope.system=all&role.id=13803fbdb99c469b9c329ce4e9d995d2 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:01.899255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.012819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2053d65-3db4-4cbb-a21b-2181a207d677 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.041468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c3ec-a25e-41e5-960b-fbf3647a51ad None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.060751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.061590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.061877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.062157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.063839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:02.065592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:02.068272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 179/368] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:02.069115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.073520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:02.090296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.090950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.092449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.092766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.110073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:02.117517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:02.118511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:02.119268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:02.127578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 190/369] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:02.127578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.154872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:02.234773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40531425-71e4-429d-973c-5b67141c1906 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.279691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.280708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.281397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.281910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.295912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5efaaf-3e49-40d0-a96d-e22802f72bc6 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.347249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.348877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.348877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.349069 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.350103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:02.351582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:02.358437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 191/370] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:02.358437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.382337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:02.384369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:02.388745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:02.513551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:02.537442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90f0a1-e465-4fe9-b469-2c4d4857144a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.543119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 180/371] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:02.544074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.574504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.575359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.575812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.576259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.577072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:02.577522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:02.579080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:02.590476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 192/372] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:02.592006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.663464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740c7b-6d8b-4cb8-b6ba-5bd8f3b8421f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.666184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:02.703377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.704685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.705062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.705432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.706356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:02.706761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:02.708455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:02.710297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc95e16-6e29-4869-8df6-047befaecef1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.714734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 181/373] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:02.715742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.749593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.750639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.751104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.751587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.753108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:02.754854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:02.756901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 193/374] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:02.762818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:02.791981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:02.897097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bc6d-60d8-4362-9b09-e87aface9105 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.956753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:02.957779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:02.958285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:02.958760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:02.962118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:02.963829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:02.966020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 194/375] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:02.966989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25cc50-c1f4-4eee-b654-4c81bd942a9c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:02.981285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:03.016994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2b96c-f7f7-443b-9147-5495f0d84cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:03.017834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2b96c-f7f7-443b-9147-5495f0d84cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.018021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2b96c-f7f7-443b-9147-5495f0d84cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.018273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2b96c-f7f7-443b-9147-5495f0d84cc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.058612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:03.092594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:03.115816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2b96c-f7f7-443b-9147-5495f0d84cc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:03.117341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2b96c-f7f7-443b-9147-5495f0d84cc0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:03.117664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc03df-5d45-4f0c-ade9-689ffdc6e016 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:03.158343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:03.159306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.159792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.160327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.161287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:03.161817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:03.163913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:03.176303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 195/376] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=491f8de5da624289a3a9b28c155d764c => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:03.267227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 182/377] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:03.273499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:03.415890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d015d-ec84-4fe6-84cf-0283cef97c2e None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.415890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d015d-ec84-4fe6-84cf-0283cef97c2e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.415890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d015d-ec84-4fe6-84cf-0283cef97c2e None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.416833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 183/378] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:03.427078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094cac-8c9b-4183-aa72-900e7d75889d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.427078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094cac-8c9b-4183-aa72-900e7d75889d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.427078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094cac-8c9b-4183-aa72-900e7d75889d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.435447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdf0e8-cc9c-496b-80fb-ee62ee87d214 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:03.465241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b8094cac-8c9b-4183-aa72-900e7d75889d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:03.466463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b8094cac-8c9b-4183-aa72-900e7d75889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:03.476431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:03.477300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.477727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.478163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] PATH_INFO: `/v3/projects/6c8b317543504cf3bafa443bc2186b42` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.479468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] RBAC: Authorizing `identity:get_project(project_id=6c8b317543504cf3bafa443bc2186b42)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:03.481089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:03.483197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 196/379] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:03 2022] GET /identity/v3/projects/6c8b317543504cf3bafa443bc2186b42 => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:03.488752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:03.506963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:03.609603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbbe7f-8a91-400b-b892-54d0b9b3d128 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:03.634791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:03.647861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:03.648858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.649287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.649738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] PATH_INFO: `/v3/groups/c1281bb0ec24410d82d5ced8b4a75a5e` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.654069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 184/380] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:03.657414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] RBAC: Authorizing `identity:get_group(group_id=c1281bb0ec24410d82d5ced8b4a75a5e)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:03.659129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:03.661324 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 197/381] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:03 2022] GET /identity/v3/groups/c1281bb0ec24410d82d5ced8b4a75a5e => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:03.667506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8094cac-8c9b-4183-aa72-900e7d75889d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:03.677881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:03.824492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:03.855060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07f226-7573-4ad7-a140-80a659d08b2c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:03.876455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8094cac-8c9b-4183-aa72-900e7d75889d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:03.919351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:03.919991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.921088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.921088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.933699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:03.940550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:03.941501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:03.942259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:03.944608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 198/382] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:03.950559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:03.954549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:03.955224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:03.955561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:03.955856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:03.956613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] RBAC: Adding query filter params (group.id=c1281bb0ec24410d82d5ced8b4a75a5e, role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=6c8b317543504cf3bafa443bc2186b42) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:03.956985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:03.958487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:03.968517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 185/383] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:03 2022] GET /identity/v3/role_assignments?group.id=c1281bb0ec24410d82d5ced8b4a75a5e&scope.project.id=6c8b317543504cf3bafa443bc2186b42&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:04.107691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686c6d-60b6-4f7b-a1c6-ca0eccd243ef None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:04.146076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9a569-3971-49c6-8fd6-4b704c3b246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:04.146979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9a569-3971-49c6-8fd6-4b704c3b246e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.147448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9a569-3971-49c6-8fd6-4b704c3b246e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.147890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9a569-3971-49c6-8fd6-4b704c3b246e None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.148756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9a569-3971-49c6-8fd6-4b704c3b246e None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:04.149243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9a569-3971-49c6-8fd6-4b704c3b246e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:04.150786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9a569-3971-49c6-8fd6-4b704c3b246e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:04.156798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:04.162556 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 199/384] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:42:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:04.168748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:04.284554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:04.340992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302a06c-6a9e-41c8-92d2-137c652af224 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:04.381799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b411d6-08a2-4559-85e8-864904e572e8 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.382159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b411d6-08a2-4559-85e8-864904e572e8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.382555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b411d6-08a2-4559-85e8-864904e572e8 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.384186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 200/385] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:04.390012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf0725-d23e-49b2-96bf-a165e6c2fd52 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.390370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf0725-d23e-49b2-96bf-a165e6c2fd52 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.390695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf0725-d23e-49b2-96bf-a165e6c2fd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.394010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:04.394660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.394945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.395222 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.399435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:04.401238 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:04.403469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 186/386] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:04 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:04.416261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:04.434727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4acf0725-d23e-49b2-96bf-a165e6c2fd52 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:04.436398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4acf0725-d23e-49b2-96bf-a165e6c2fd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:04.572466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 201/387] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:04.595957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf0725-d23e-49b2-96bf-a165e6c2fd52 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:04.606920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d9c78-1e5d-4f24-9c62-8701e65ff943 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:04.658758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:04.673205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:04.673939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.674227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.674505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.681176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:04.682685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:04.684656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 187/388] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:04 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:04.692694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:04.715557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf0725-d23e-49b2-96bf-a165e6c2fd52 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:04.747412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:04.755005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:04.756018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.756990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.768080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:04.773165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:04.774024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:04.774791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:04.777335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 202/389] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:04.783570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:04.894930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd671a0f-6967-4a16-a534-a974fe8dac5c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:04.895613 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087171a-68ff-4c1c-9a40-28dfd9b3a630 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:04.953282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:04.954626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.954904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.955233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.955915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:04.956246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:04.957626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:04.966662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 203/390] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:04.972672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:04.985425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:04.986113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:04.986392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:04.986668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:04.993389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=906d21dfa6d74405b8c0b3b5558f5c88, group_id=0944d71bc6ae4cb496d04abc2d2728e4, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:04.994827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:05.036335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:05.039189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 188/391] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 10 16:42:04 2022] PUT /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:42:05.093631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9e865-b109-46b8-83ee-479973973963 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:05.133515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:05.134563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.135044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.135521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.137104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:05.138795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:05.141569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 204/392] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:04 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:05.147786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:05.224476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:05.293824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474da07-571c-4c78-9565-187cfe5f3b3f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:05.333033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505e01c-516f-45fb-8c50-681a25acd572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:05.334039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505e01c-516f-45fb-8c50-681a25acd572 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.334650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505e01c-516f-45fb-8c50-681a25acd572 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.335177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505e01c-516f-45fb-8c50-681a25acd572 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.336644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505e01c-516f-45fb-8c50-681a25acd572 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:05.338281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505e01c-516f-45fb-8c50-681a25acd572 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:05.340130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 189/393] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:05 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:05.346188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:05.466631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec45b7-4927-407d-bd46-dfb6277590d4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:05.488886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:05.514090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:05.515068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.515348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.515622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.522022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] RBAC: Authorizing `identity:create_grant(project_id=906d21dfa6d74405b8c0b3b5558f5c88, group_id=0944d71bc6ae4cb496d04abc2d2728e4, role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:05.523989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:05.524211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb191cf-21dd-4e39-84b7-076431acf0be None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.524468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb191cf-21dd-4e39-84b7-076431acf0be None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.524751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb191cf-21dd-4e39-84b7-076431acf0be None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.526170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 190/394] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:05.533414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447eb2b-7613-49d1-ba35-3bdd2b4221d0 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.533861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447eb2b-7613-49d1-ba35-3bdd2b4221d0 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.534261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447eb2b-7613-49d1-ba35-3bdd2b4221d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.541814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 205/395] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 10 16:42:05 2022] PUT /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/491f8de5da624289a3a9b28c155d764c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:05.580232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f447eb2b-7613-49d1-ba35-3bdd2b4221d0 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:05.580232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f447eb2b-7613-49d1-ba35-3bdd2b4221d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:05.695613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 191/396] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:05.731401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:05.834917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b1ec9-a2d5-49fb-aa02-b289270c39fd None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.835159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b1ec9-a2d5-49fb-aa02-b289270c39fd None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.835389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b1ec9-a2d5-49fb-aa02-b289270c39fd None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.837213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 192/397] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:05.842053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164227-d21e-4338-bf7e-1b49f0dd1540 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.842343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164227-d21e-4338-bf7e-1b49f0dd1540 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.842622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164227-d21e-4338-bf7e-1b49f0dd1540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.859046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e72fbb-62df-4975-bbf7-f926ae69048f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:05.864086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-28164227-d21e-4338-bf7e-1b49f0dd1540 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:05.865359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-28164227-d21e-4338-bf7e-1b49f0dd1540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:05.903597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:05.905119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:05.905688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:05.906261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:05.921095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:05.929155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:05.930657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:05.937302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 206/398] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:05.939260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:05.942941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:05.984399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 193/399] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:05.996321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:06.008692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28164227-d21e-4338-bf7e-1b49f0dd1540 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.068264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bb6be-db34-49e9-9e84-c89de6e7647a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.109683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.111273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.111754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.112274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.113146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:06.113732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:06.115763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:06.131307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 207/400] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:42:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:06.133752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.202329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28164227-d21e-4338-bf7e-1b49f0dd1540 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.244168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d5b78-62dc-428d-811f-deabb5791da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.245198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5b78-62dc-428d-811f-deabb5791da0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.245743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5b78-62dc-428d-811f-deabb5791da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.246227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5b78-62dc-428d-811f-deabb5791da0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.258057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d5b78-62dc-428d-811f-deabb5791da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:06.258598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03182966-86b1-4fd8-ace6-0ddfbeba893a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.260168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-387d5b78-62dc-428d-811f-deabb5791da0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:06.264546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:06.265438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:06.266270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:06.269672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 194/401] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:06.272249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256de34-3b6e-499e-9bfe-da94810f075b None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.273005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256de34-3b6e-499e-9bfe-da94810f075b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.273491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256de34-3b6e-499e-9bfe-da94810f075b None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.276183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 195/402] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:06.277295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256de34-3b6e-499e-9bfe-da94810f075b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.318306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34953eb3-7bb1-4eb5-86c4-d202f1838999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.319358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953eb3-7bb1-4eb5-86c4-d202f1838999 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.319869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953eb3-7bb1-4eb5-86c4-d202f1838999 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.320218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953eb3-7bb1-4eb5-86c4-d202f1838999 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.321824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34953eb3-7bb1-4eb5-86c4-d202f1838999 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:06.323610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34953eb3-7bb1-4eb5-86c4-d202f1838999 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:06.330093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 208/403] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:06.331447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026e19-0b34-42c1-8fa1-f9b3a27cd8cc None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.331739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026e19-0b34-42c1-8fa1-f9b3a27cd8cc None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.332022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026e19-0b34-42c1-8fa1-f9b3a27cd8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.342411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:06.352499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-0b026e19-0b34-42c1-8fa1-f9b3a27cd8cc None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:06.353656 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0b026e19-0b34-42c1-8fa1-f9b3a27cd8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:06.423311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256de34-3b6e-499e-9bfe-da94810f075b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.448289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 209/404] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:06.449343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b026e19-0b34-42c1-8fa1-f9b3a27cd8cc None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.470003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.471338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.471748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.472206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.473053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:06.473502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:06.475109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:06.486767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 196/405] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:06.487835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.572400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:06.575785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b026e19-0b34-42c1-8fa1-f9b3a27cd8cc None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.614066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.614840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.615230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.615667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.616931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:06.618551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:06.620446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 210/406] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:06.621424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.624716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f3a2-9e3f-47d3-a0e6-c79ef235a8dd None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.637016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:06.670007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.670848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.671653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.672091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.685062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:06.692196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:06.693234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:06.694195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:06.696992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 197/407] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:06.697918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.748497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf862-7a15-4ef6-ab80-bd6592b6f9c2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.790187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.790978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.791423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.791805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.793563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:06.794604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:06.796497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 211/408] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:06.797444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.831074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5bddc-f5e6-4d66-b2d8-db2ee2032acb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.836714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:06.872728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:06.874136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:06.874823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:06.875240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:06.878799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d, group_id=0944d71bc6ae4cb496d04abc2d2728e4, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:06.880479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:06.898792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 198/409] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 10 16:42:06 2022] PUT /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:06.899959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:06.937522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4779b-c9e4-4e0c-99e8-d32b2812ff91 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:06.958821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:07.026735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:07.032716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.033360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.033775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.034593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:07.035067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:07.036717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:07.054244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 212/410] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:07.060032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:07.085562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbea3c-6917-4643-8572-73ade7d42638 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:07.131929 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:07.132612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.132884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.133157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.135715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:07.137099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:07.139751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 199/411] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:06 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:07.145968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:07.185714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:07.190944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4e13-311a-4c1c-aef4-363d74f5b513 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:07.226899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:07.227907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.228442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.228915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] PATH_INFO: `/v3/groups/c1281bb0ec24410d82d5ced8b4a75a5e` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.230348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] RBAC: Authorizing `identity:get_group(group_id=c1281bb0ec24410d82d5ced8b4a75a5e)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:07.232232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:07.234542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 213/412] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:07 2022] GET /identity/v3/groups/c1281bb0ec24410d82d5ced8b4a75a5e => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:07.241015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:07.305296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:07.316509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195dd4f-d0cc-485c-9684-f33f81930b54 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:07.350596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274b385-33f9-46d9-b4a4-38b45f186472 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:07.387028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:07.388001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.388493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.388954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] PATH_INFO: `/v3/projects/6c8b317543504cf3bafa443bc2186b42` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.390370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] RBAC: Authorizing `identity:get_project(project_id=6c8b317543504cf3bafa443bc2186b42)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:07.392170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:07.394403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 214/413] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:07 2022] GET /identity/v3/projects/6c8b317543504cf3bafa443bc2186b42 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:07.400491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:07.406074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bbe53-9664-4b63-9c33-f824c1f6e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:07.409478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbe53-9664-4b63-9c33-f824c1f6e86e None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.409478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbe53-9664-4b63-9c33-f824c1f6e86e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.409478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbe53-9664-4b63-9c33-f824c1f6e86e None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.417060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bbe53-9664-4b63-9c33-f824c1f6e86e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d, group_id=0944d71bc6ae4cb496d04abc2d2728e4, role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:07.418533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bbe53-9664-4b63-9c33-f824c1f6e86e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:07.448879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:07.458954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 200/414] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 10 16:42:07 2022] PUT /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d/groups/0944d71bc6ae4cb496d04abc2d2728e4/roles/491f8de5da624289a3a9b28c155d764c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:07.523924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b453ea-f1f3-4660-b7e1-e73f21b1d0ff None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:07.562109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b05952-b9e5-456c-8505-38a304253233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:07.563197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b05952-b9e5-456c-8505-38a304253233 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.563671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b05952-b9e5-456c-8505-38a304253233 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.564138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b05952-b9e5-456c-8505-38a304253233 None admin] PATH_INFO: `/v3/projects/6c8b317543504cf3bafa443bc2186b42/groups/c1281bb0ec24410d82d5ced8b4a75a5e/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.566672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b05952-b9e5-456c-8505-38a304253233 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c8b317543504cf3bafa443bc2186b42, group_id=c1281bb0ec24410d82d5ced8b4a75a5e, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:07.568412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b05952-b9e5-456c-8505-38a304253233 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:07.590237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 215/415] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 10 16:42:07 2022] PUT /identity/v3/projects/6c8b317543504cf3bafa443bc2186b42/groups/c1281bb0ec24410d82d5ced8b4a75a5e/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:42:07.798864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef67055-9259-452c-963c-1845a502f74d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.798864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef67055-9259-452c-963c-1845a502f74d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.799088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef67055-9259-452c-963c-1845a502f74d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.800670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 201/416] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:07.806097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d317e1-6b66-4f11-8a54-c43a485393ea None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.806384 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d317e1-6b66-4f11-8a54-c43a485393ea None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.806663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d317e1-6b66-4f11-8a54-c43a485393ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.830522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-25d317e1-6b66-4f11-8a54-c43a485393ea None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:07.831721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-25d317e1-6b66-4f11-8a54-c43a485393ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:07.901406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d35c36-19f0-44c0-a1b6-29ec269d7e41 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.902058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d35c36-19f0-44c0-a1b6-29ec269d7e41 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.902535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d35c36-19f0-44c0-a1b6-29ec269d7e41 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.908428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 202/417] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:07.909823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6b0c0-2c2b-46fe-9019-19b9cc8a4605 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:07.910118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6b0c0-2c2b-46fe-9019-19b9cc8a4605 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:07.910398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6b0c0-2c2b-46fe-9019-19b9cc8a4605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:07.918763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 216/418] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:07.940133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d317e1-6b66-4f11-8a54-c43a485393ea None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:07.942029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d9e6b0c0-2c2b-46fe-9019-19b9cc8a4605 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:07.948178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d9e6b0c0-2c2b-46fe-9019-19b9cc8a4605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:07.996553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:08.062413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d317e1-6b66-4f11-8a54-c43a485393ea None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:08.101616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:08.102397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:08.102676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:08.102942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:08.114191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:08.119471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:08.120388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:08.121305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 217/419] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:08.126643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:08.152569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 203/420] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:08.178083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6b0c0-2c2b-46fe-9019-19b9cc8a4605 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:08.290025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:08.318562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:08.340978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6b0c0-2c2b-46fe-9019-19b9cc8a4605 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:08.346504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c129a-c529-4343-b131-101f9148f93d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:08.404443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:08.412015 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:08.413991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:08.414434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:08.426426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:08.427232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:08.427799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:08.429202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:08.430029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:08.430653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:08.432178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:08.442005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:08.450873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:08.451926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:08.452935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:08.455786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 218/421] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:42:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:08.456801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:08.457170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 204/422] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:08.465192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:08.641823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a84ba-9ade-4dd4-b7c6-43e74ad4db08 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:08.658260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb3a4-f79e-4c65-af5e-5dbffaa6e1ef None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:08.709009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:08.710778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:08.712024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:08.712485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:08.714426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:08.716018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:08.717541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:08.728279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:08.729088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:08.729511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:08.730062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:08.733806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 205/423] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:08.744067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:08.752269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:08.762572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:08.811908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:08.814199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:08.926214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 219/424] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:08.933872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:08.969795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a53d2-c023-4829-bc69-0a7df50f8d1b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.024588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:09.046436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c294af-092b-4783-8df3-d6e91c5b1965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.047915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c294af-092b-4783-8df3-d6e91c5b1965 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.050034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c294af-092b-4783-8df3-d6e91c5b1965 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.051821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c294af-092b-4783-8df3-d6e91c5b1965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.134114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4032ad-d9fc-4a03-ae45-fd49aa864daa None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.174038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c294af-092b-4783-8df3-d6e91c5b1965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:09.179136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c294af-092b-4783-8df3-d6e91c5b1965 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:09.199047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:09.208598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.209446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.211773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.212445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.214902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:09.217550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:09.224497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 220/425] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:08 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:09.225449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:09.326768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 206/426] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:09.328545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6b338-060a-4b71-8e1a-e2f15b691a29 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.329298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6b338-060a-4b71-8e1a-e2f15b691a29 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.329737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6b338-060a-4b71-8e1a-e2f15b691a29 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.332307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 207/427] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:09.333162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6b338-060a-4b71-8e1a-e2f15b691a29 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:09.381591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70276-4009-480c-81ee-ac10ce3053c2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.404555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:09.418490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53586d03-fc6e-44ff-b0b9-7cd58cabc3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.419365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53586d03-fc6e-44ff-b0b9-7cd58cabc3ef None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.419869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53586d03-fc6e-44ff-b0b9-7cd58cabc3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.420282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53586d03-fc6e-44ff-b0b9-7cd58cabc3ef None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.421841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53586d03-fc6e-44ff-b0b9-7cd58cabc3ef None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:09.424401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53586d03-fc6e-44ff-b0b9-7cd58cabc3ef None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:09.426580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 221/428] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:09 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:09.429475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67179938-97b6-427a-90c1-5ccdb0578e3b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.429925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67179938-97b6-427a-90c1-5ccdb0578e3b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.430353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67179938-97b6-427a-90c1-5ccdb0578e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.450190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-67179938-97b6-427a-90c1-5ccdb0578e3b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:09.451779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-67179938-97b6-427a-90c1-5ccdb0578e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:09.476549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6b338-060a-4b71-8e1a-e2f15b691a29 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.486580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:09.513560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.514302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.514660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.515065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] PATH_INFO: `/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.517066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] RBAC: Authorizing `identity:get_project(project_id=906d21dfa6d74405b8c0b3b5558f5c88)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:09.518591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:09.520674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 208/429] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:09 2022] GET /identity/v3/projects/906d21dfa6d74405b8c0b3b5558f5c88 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:09.521642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:09.531516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 222/430] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:09.533199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67179938-97b6-427a-90c1-5ccdb0578e3b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:09.640767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67179938-97b6-427a-90c1-5ccdb0578e3b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.651646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed5b5-cdd7-4b97-9c7a-24a6bc455906 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.656574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:09.681607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.682640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.683218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.683702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.685222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:09.687320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:09.687915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.688612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.688892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.689216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.690322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:09.690631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:09.691040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 223/431] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:09 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:09.692062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:09.692182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:09.698448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 209/432] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:09 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:09.705024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:09.783094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:09.810350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279f934-f028-43a1-a46b-000ce585982d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.849493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae4b86-ecac-4189-bb6a-4a0305ae9f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.850558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae4b86-ecac-4189-bb6a-4a0305ae9f4d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.851035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae4b86-ecac-4189-bb6a-4a0305ae9f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.851500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae4b86-ecac-4189-bb6a-4a0305ae9f4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.864193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae4b86-ecac-4189-bb6a-4a0305ae9f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-99ae4b86-ecac-4189-bb6a-4a0305ae9f4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:09.869431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:09.870328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:42:09.871113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:09.873489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 224/433] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:09.886308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b49722-4d55-4ee0-aedd-4d02d27e0c80 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.886614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b49722-4d55-4ee0-aedd-4d02d27e0c80 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.888511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b49722-4d55-4ee0-aedd-4d02d27e0c80 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.888630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1e170-3f13-4bd0-9110-4cf78df271b0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:09.888717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 225/434] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:09.894397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1273f-c1b1-4219-b6b3-dd7aa54d20c6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.894687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1273f-c1b1-4219-b6b3-dd7aa54d20c6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.895072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1273f-c1b1-4219-b6b3-dd7aa54d20c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.914637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-fcb1273f-c1b1-4219-b6b3-dd7aa54d20c6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:09.915844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fcb1273f-c1b1-4219-b6b3-dd7aa54d20c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:09.926958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:09.927587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:09.927934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:09.928239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:09.929037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:09.929350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:09.930881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:09.937988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 210/435] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:09 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=906d21dfa6d74405b8c0b3b5558f5c88&role.id=491f8de5da624289a3a9b28c155d764c => generated 611 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:09.938809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:09.957704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.006679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 226/436] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:10.007983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959097a-0ad8-419a-a84f-49bba5feaa64 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.008290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959097a-0ad8-419a-a84f-49bba5feaa64 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.008569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959097a-0ad8-419a-a84f-49bba5feaa64 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.010007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 227/437] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:10.017147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d3e2-52bd-4399-b967-b2b6dee64be6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.017147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d3e2-52bd-4399-b967-b2b6dee64be6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.017147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d3e2-52bd-4399-b967-b2b6dee64be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.036852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9657d3e2-52bd-4399-b967-b2b6dee64be6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:10.038244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9657d3e2-52bd-4399-b967-b2b6dee64be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:10.082185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a541-145c-41a4-95ac-bac7e7280eda None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.086497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.132781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.133403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.133575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 228/438] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:10.133669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.133949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.134454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657d3e2-52bd-4399-b967-b2b6dee64be6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.134897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] RBAC: Adding query filter params (name=member) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:10.134897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:10.137307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:10.147772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 211/439] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:42:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:10.154471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.270139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.273721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657d3e2-52bd-4399-b967-b2b6dee64be6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.277655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefaa47-e877-4ee2-9e98-e22c3bce7917 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.318773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.319423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.319703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.319971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.322224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.323004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.323764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.324005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.341115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:10.347251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:10.348488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:10.349276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 229/440] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:10.350018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.364233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.403166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:10.404684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:10.474065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bfafc-db26-4cb3-979b-03eccbea4608 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.507498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.508133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.508442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.508716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.519316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:10.524554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 212/441] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:42:10.524554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:10.524757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:10.525580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 230/442] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:10.526804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.548958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.587116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.621100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03320a3-654d-490b-bad4-b551d2d1c350 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.629067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6486162-b71d-4405-aa5a-35b8773d82d8 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.655372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.655984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.656307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.656585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.657251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:10.657552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:10.658915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:10.662289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.662907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.663182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.663453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.664587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:10.666013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:10.666606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 213/443] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:10.667390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.667883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 231/444] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:10.672109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.760280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.768903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2175c-74d6-4a6e-ac05-6468271d3643 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.774255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb65e2e-ed57-465d-a94b-4e81c7f7a010 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.794460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.802814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425a836-f382-4b73-80ce-40866566389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.803428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425a836-f382-4b73-80ce-40866566389d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.803737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425a836-f382-4b73-80ce-40866566389d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.804013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425a836-f382-4b73-80ce-40866566389d None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.804715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425a836-f382-4b73-80ce-40866566389d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:10.805025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425a836-f382-4b73-80ce-40866566389d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:10.806415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425a836-f382-4b73-80ce-40866566389d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:10.807568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.808184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.808469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.808741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.815121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 214/445] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:10.815914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425a836-f382-4b73-80ce-40866566389d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.881057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:10.882709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:10.911437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425a836-f382-4b73-80ce-40866566389d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:10.945121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:10.945823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:10.946095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:10.946369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:10.947478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:10.948928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:10.955285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 215/446] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:10.955285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:10.968572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:10.981548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 232/447] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:42:10.985571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:11.010064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:11.051063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44b6df-9278-4003-874c-2a77e6504173 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.083637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d089a6-58e2-4b89-9e13-a9a1e29b6608 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.084007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.084645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.084925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.085197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.117083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.117672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.117947 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.118216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.118924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=230b13c1aa214ed4bb560f74e036a7b7, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:11.119234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.120708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.126844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 233/448] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=230b13c1aa214ed4bb560f74e036a7b7 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:11.127599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:11.157295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.158789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.189574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:11.225930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:11.246523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237bf7c-d80b-46d3-a39a-ab626f03d3f1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.282230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 216/449] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:11.283283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:11.288310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.288952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.289233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.289504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] PATH_INFO: `/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.290622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf9ec77de774515a68ae9f6b1d5ff2d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.292061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.293722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 234/450] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:11 2022] GET /identity/v3/projects/dcf9ec77de774515a68ae9f6b1d5ff2d => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:11.300424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:11.402923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60579d8-cdec-4254-8101-4b199537bc64 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.427996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e658-ff9d-47fd-89f6-ec938e3826a1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.445081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.445996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.446480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.446928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] PATH_INFO: `/v3/projects/6c8b317543504cf3bafa443bc2186b42` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.448064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] RBAC: Authorizing `identity:get_project(project_id=6c8b317543504cf3bafa443bc2186b42)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.449518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.450018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:11.453682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 217/451] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:42:11 2022] GET /identity/v3/projects/6c8b317543504cf3bafa443bc2186b42 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:11.457832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:11.468082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.468082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.468082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.468420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] PATH_INFO: `/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.470010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] RBAC: Authorizing `identity:get_group(group_id=0944d71bc6ae4cb496d04abc2d2728e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.471491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.473407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 235/452] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:11 2022] GET /identity/v3/groups/0944d71bc6ae4cb496d04abc2d2728e4 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:11.479108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:11.492608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:11.566681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d63ac5-3886-41e7-ad8c-d9bce6db20d9 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.581373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbad363-264c-4a98-94c0-efaeb94617c1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.599166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.599786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.600062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.600429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] PATH_INFO: `/v3/groups/c1281bb0ec24410d82d5ced8b4a75a5e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.601535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] RBAC: Authorizing `identity:get_group(group_id=c1281bb0ec24410d82d5ced8b4a75a5e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.602961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.604556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 218/453] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 10 16:42:11 2022] GET /identity/v3/groups/c1281bb0ec24410d82d5ced8b4a75a5e => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:11.610977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:11.614872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1d22-d9e5-4143-a89f-f1650892bc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.615478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1d22-d9e5-4143-a89f-f1650892bc79 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.615764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1d22-d9e5-4143-a89f-f1650892bc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.616039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1d22-d9e5-4143-a89f-f1650892bc79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.616831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef1d22-d9e5-4143-a89f-f1650892bc79 None admin] RBAC: Adding query filter params (group.id=0944d71bc6ae4cb496d04abc2d2728e4, role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:11.617147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef1d22-d9e5-4143-a89f-f1650892bc79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.618569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef1d22-d9e5-4143-a89f-f1650892bc79 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.624315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 236/454] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:11 2022] GET /identity/v3/role_assignments?group.id=0944d71bc6ae4cb496d04abc2d2728e4&scope.project.id=dcf9ec77de774515a68ae9f6b1d5ff2d&role.id=491f8de5da624289a3a9b28c155d764c => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:11.708420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:11.715553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640180d-683b-450d-980d-0341f834bcd6 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:11.750608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef9977-97a6-4c6c-b74c-e41029fc71dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:11.751229 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef9977-97a6-4c6c-b74c-e41029fc71dc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:11.751509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef9977-97a6-4c6c-b74c-e41029fc71dc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:11.751781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef9977-97a6-4c6c-b74c-e41029fc71dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:11.752546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef9977-97a6-4c6c-b74c-e41029fc71dc None admin] RBAC: Adding query filter params (group.id=c1281bb0ec24410d82d5ced8b4a75a5e, role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=6c8b317543504cf3bafa443bc2186b42) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:11.752856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef9977-97a6-4c6c-b74c-e41029fc71dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:11.754305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef9977-97a6-4c6c-b74c-e41029fc71dc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:11.760649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 219/455] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 10 16:42:11 2022] GET /identity/v3/role_assignments?group.id=c1281bb0ec24410d82d5ced8b4a75a5e&scope.project.id=6c8b317543504cf3bafa443bc2186b42&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:14.341704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e7734-6984-41a6-aef5-2e2a35dc6b1a None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.353228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e7734-6984-41a6-aef5-2e2a35dc6b1a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.353228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e7734-6984-41a6-aef5-2e2a35dc6b1a None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.353228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 237/456] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:14.356179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1950b5b-6185-44de-ad0f-fe94bebfe304 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.356179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1950b5b-6185-44de-ad0f-fe94bebfe304 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.356179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1950b5b-6185-44de-ad0f-fe94bebfe304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.381345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c1950b5b-6185-44de-ad0f-fe94bebfe304 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:14.382892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c1950b5b-6185-44de-ad0f-fe94bebfe304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:14.478352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 220/457] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:14.517467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e7734-6984-41a6-aef5-2e2a35dc6b1a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:14.556926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059881d8-0663-4599-9a9c-da5a29d69dae None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.557600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059881d8-0663-4599-9a9c-da5a29d69dae None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.558090 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059881d8-0663-4599-9a9c-da5a29d69dae None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.559646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 221/458] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:14.565526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c20439-c263-4caf-9f5a-3ee002249a1e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.566125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c20439-c263-4caf-9f5a-3ee002249a1e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.566560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c20439-c263-4caf-9f5a-3ee002249a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.592264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-03c20439-c263-4caf-9f5a-3ee002249a1e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:14.594927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-03c20439-c263-4caf-9f5a-3ee002249a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:14.613025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:14.693300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e7734-6984-41a6-aef5-2e2a35dc6b1a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:14.708704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 222/459] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:14.716170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05c4dc-dd12-4209-85c5-6e79db254882 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.716666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05c4dc-dd12-4209-85c5-6e79db254882 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.717089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05c4dc-dd12-4209-85c5-6e79db254882 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.718770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 223/460] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:14.724630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0e29-d31b-4090-862b-e22c6c1242a3 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.725086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0e29-d31b-4090-862b-e22c6c1242a3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.725506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0e29-d31b-4090-862b-e22c6c1242a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.737214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:14.738149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.738632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.739128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.750740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:14.752579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:14.756991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-18fe0e29-d31b-4090-862b-e22c6c1242a3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:14.758248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-18fe0e29-d31b-4090-862b-e22c6c1242a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:14.770181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:14.771125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:14.771996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:14.771996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:14.771996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:14.773250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:14.774675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 238/461] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:14.775760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:14.836047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 224/462] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:14.837137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0e29-d31b-4090-862b-e22c6c1242a3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:14.914423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc657481-f217-4c02-89a7-2a7547e8166c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:14.917797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:14.952971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:14.954957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:14.955428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:14.955868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:14.968673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:14.971084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:14.979528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0e29-d31b-4090-862b-e22c6c1242a3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:14.987308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:14.988191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:14.989017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:14.989017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:14.989017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:14.995798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 239/463] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:14.997023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.021287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.021919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.022204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.022485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.023145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:15.023452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.025407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.041634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 225/464] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:15.042494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.056769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:15.114719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440090e-3630-4649-a6cc-e78afd94804d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:15.150407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.151040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.151322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.151590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.162329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.163794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.165203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde161-b329-4dee-bb9b-43cb5ce7a89a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:15.178700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:15.179595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:15.180448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:15.180448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:15.180448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:15.180448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 240/465] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:15.180671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.193330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:15.207496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03c40e-3da6-4b7a-bd10-e109f0dac20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.208430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03c40e-3da6-4b7a-bd10-e109f0dac20e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.208718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03c40e-3da6-4b7a-bd10-e109f0dac20e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.208996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03c40e-3da6-4b7a-bd10-e109f0dac20e None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.209801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03c40e-3da6-4b7a-bd10-e109f0dac20e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:15.210443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03c40e-3da6-4b7a-bd10-e109f0dac20e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.212284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03c40e-3da6-4b7a-bd10-e109f0dac20e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.226432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 226/466] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:15.230967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e7426-1573-4333-bbf2-3cc866f94e9a None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.230967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e7426-1573-4333-bbf2-3cc866f94e9a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.230967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e7426-1573-4333-bbf2-3cc866f94e9a None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.230967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 227/467] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:15.230967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e7426-1573-4333-bbf2-3cc866f94e9a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.300522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:15.302011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11fc66-f01c-4612-b57b-23678ac6442d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:15.338163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.338518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e7426-1573-4333-bbf2-3cc866f94e9a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:15.338773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.339054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.339327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.339967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.341501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.363916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 241/468] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:42:15 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:15.373178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.380368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df0858-6723-4442-bd93-e5ef715725e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.380913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df0858-6723-4442-bd93-e5ef715725e3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.381401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df0858-6723-4442-bd93-e5ef715725e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.381401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df0858-6723-4442-bd93-e5ef715725e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.382210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df0858-6723-4442-bd93-e5ef715725e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:15.382584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df0858-6723-4442-bd93-e5ef715725e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.384102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df0858-6723-4442-bd93-e5ef715725e3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.403292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 228/469] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:15.404870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1538d-fd51-4b66-b807-7de53957a857 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.405234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1538d-fd51-4b66-b807-7de53957a857 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.405909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1538d-fd51-4b66-b807-7de53957a857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.436845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-95b1538d-fd51-4b66-b807-7de53957a857 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:15.439168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-95b1538d-fd51-4b66-b807-7de53957a857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:15.469287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:15.517576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970332-cccf-425b-b293-e1427bd74445 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:15.529630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 229/470] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:15.530306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1538d-fd51-4b66-b807-7de53957a857 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.556193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e98f8-8626-475d-b626-9cb4d06b1cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.557056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e98f8-8626-475d-b626-9cb4d06b1cfd None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.557491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e98f8-8626-475d-b626-9cb4d06b1cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.557920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e98f8-8626-475d-b626-9cb4d06b1cfd None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.558574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e98f8-8626-475d-b626-9cb4d06b1cfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.560019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e98f8-8626-475d-b626-9cb4d06b1cfd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.584519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 242/471] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:42:15 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:15.585798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86ae40-1077-42b6-bd8d-90d4bc810e36 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.586091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86ae40-1077-42b6-bd8d-90d4bc810e36 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.586370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86ae40-1077-42b6-bd8d-90d4bc810e36 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.587861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 243/472] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:15.588640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86ae40-1077-42b6-bd8d-90d4bc810e36 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.601423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:15.653161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1538d-fd51-4b66-b807-7de53957a857 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:15.694458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548bbba-5be0-447f-ab98-fd38b39f22b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.695477 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548bbba-5be0-447f-ab98-fd38b39f22b0 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.696030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548bbba-5be0-447f-ab98-fd38b39f22b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.696529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548bbba-5be0-447f-ab98-fd38b39f22b0 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.697393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548bbba-5be0-447f-ab98-fd38b39f22b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.699060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548bbba-5be0-447f-ab98-fd38b39f22b0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.718670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86ae40-1077-42b6-bd8d-90d4bc810e36 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:15.729579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 230/473] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:42:15 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:15.732415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb7097-4025-4ab1-87dc-e8708c247f17 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.733139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb7097-4025-4ab1-87dc-e8708c247f17 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.733749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb7097-4025-4ab1-87dc-e8708c247f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.753255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:15.772959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f6bb7097-4025-4ab1-87dc-e8708c247f17 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:15.774189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f6bb7097-4025-4ab1-87dc-e8708c247f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:15.781368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:15.783301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:15.783301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:15.783301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:15.799899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:15.801822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:15.820546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:15.821506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:15.822367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:15.822367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:15.822367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:15.828208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 244/474] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:15.831425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.882782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 231/475] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:15.906156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb7097-4025-4ab1-87dc-e8708c247f17 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:15.972422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:15.977003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61930-2fc8-42b8-997c-96572f4c77c9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:16.037022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:16.037943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:16.038288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:16.038996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:16.039807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:16.040120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:16.041933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:16.045970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb7097-4025-4ab1-87dc-e8708c247f17 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:16.062159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 245/476] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:16.069213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:16.097630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:16.098631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:16.099896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:16.100317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:16.101836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:16.104972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:16.124028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.sql.core [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 16:42:16.124028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 16:42:16.124028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [parameters: {'id': 'f48b254482c34399bae51098995194dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 16:42:16.124028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81539) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 16:42:16.144682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result.read() Aug 10 16:42:16.150092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:42:16.151367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application yield resource Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise self.value Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application session.commit() Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 16:42:16.153065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.session.flush() Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application compat.raise_( Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 16:42:16.154570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:42:16.156117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result.read() Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:42:16.158701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [parameters: {'id': 'f48b254482c34399bae51098995194dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:16.165644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 16:42:16.167332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:16.169008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 232/477] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 10 16:42:15 2022] POST /identity/v3/roles => generated 144 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:16.169008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:16.280486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558add4-ff0f-4e57-8822-c49f9afce56b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:16.292852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:16.305542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250efc21-afb7-4baf-9ad4-81f8e9c1e86f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:16.349509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:16.350161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:16.350442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:16.350716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:16.355455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae1549-a83a-4f66-8b86-ee314c452a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:16.356067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae1549-a83a-4f66-8b86-ee314c452a2a None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:16.356367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae1549-a83a-4f66-8b86-ee314c452a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:16.356638 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae1549-a83a-4f66-8b86-ee314c452a2a None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:16.357282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae1549-a83a-4f66-8b86-ee314c452a2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:16.358723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae1549-a83a-4f66-8b86-ee314c452a2a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:16.375189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:16.386423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:16.387841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:16.388729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 233/478] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:16.394108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:16.409760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 246/479] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:42:16 2022] POST /identity/v3/users => generated 249 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:16.563899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd743979-d644-4059-950b-d43b21a54901 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:16.605523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290dde5-3cd8-47ea-9b9f-40e1490e4d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:16.606149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290dde5-3cd8-47ea-9b9f-40e1490e4d6f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:16.606424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290dde5-3cd8-47ea-9b9f-40e1490e4d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:16.606698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290dde5-3cd8-47ea-9b9f-40e1490e4d6f None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:16.607371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290dde5-3cd8-47ea-9b9f-40e1490e4d6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:16.607679 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290dde5-3cd8-47ea-9b9f-40e1490e4d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:16.609221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290dde5-3cd8-47ea-9b9f-40e1490e4d6f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:16.624321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 234/480] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:17.894126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b892d6-89e9-423f-bcff-4fdbc040c09e None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:17.894633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b892d6-89e9-423f-bcff-4fdbc040c09e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:17.894706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b892d6-89e9-423f-bcff-4fdbc040c09e None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:17.896285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 247/481] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:17.901287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7f052-bc8a-465a-985b-7197a084b125 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:17.901704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7f052-bc8a-465a-985b-7197a084b125 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:17.901871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7f052-bc8a-465a-985b-7197a084b125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:17.928350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-95e7f052-bc8a-465a-985b-7197a084b125 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:17.929565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-95e7f052-bc8a-465a-985b-7197a084b125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:17.961401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:18.007675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 235/482] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:18.045672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b892d6-89e9-423f-bcff-4fdbc040c09e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.153808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:18.172981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b892d6-89e9-423f-bcff-4fdbc040c09e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:18.227216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:18.227854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.228133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.228436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.239633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:18.244406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:18.245272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:18.246049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:18.246714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da0aaf-da27-45a0-a1b4-602791496a6c None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.246714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da0aaf-da27-45a0-a1b4-602791496a6c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.246834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 248/483] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:18.246895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da0aaf-da27-45a0-a1b4-602791496a6c None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.247471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 236/484] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:18.252283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.253008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67519-d4ec-47c9-9bb5-52be94e99092 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.253295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67519-d4ec-47c9-9bb5-52be94e99092 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.253573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67519-d4ec-47c9-9bb5-52be94e99092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.285797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-fbe67519-d4ec-47c9-9bb5-52be94e99092 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:18.286996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fbe67519-d4ec-47c9-9bb5-52be94e99092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:18.409426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 237/485] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:18.410791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e96700-d7af-4220-be7b-584b8a1a2252 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.411158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e96700-d7af-4220-be7b-584b8a1a2252 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.411604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e96700-d7af-4220-be7b-584b8a1a2252 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.413764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 238/486] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:18.423282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef81a74-1a59-4f0c-93df-17744e6807bb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:18.429618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e642d61-bd8a-4389-855e-8e2d0cc1e878 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.429919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e642d61-bd8a-4389-855e-8e2d0cc1e878 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.430201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e642d61-bd8a-4389-855e-8e2d0cc1e878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.455061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9e642d61-bd8a-4389-855e-8e2d0cc1e878 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:18.456276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9e642d61-bd8a-4389-855e-8e2d0cc1e878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:18.460944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:18.461561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.461844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.462117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.462793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:18.463096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:18.464650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:18.478621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 249/487] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:18.479499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.493983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:18.536253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 239/488] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:18.537191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e642d61-bd8a-4389-855e-8e2d0cc1e878 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.574522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:18.600502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf610e-f5f8-496e-8c50-e3cd182eb42c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:18.636073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:18.636740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.637022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.637295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.647801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:18.652989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:18.654070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:18.655018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 250/489] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:18.660290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.668917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e642d61-bd8a-4389-855e-8e2d0cc1e878 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:18.707129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:18.708077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.708077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.708393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.719230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:18.720900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:18.736803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:18.737944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:18.738989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:18.738989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:18.738989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:18.742533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 240/490] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:18.743752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.746915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:18.789754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eec8aa-3c78-4125-b676-624a04f7bc8e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:18.828707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:18.829586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.830137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.830552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.831255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:18.845071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:18.850798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:18.851691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:18.852707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:18.854797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 251/491] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:18.855775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.870796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc107fe-ba2d-42fd-b29c-4386099bb603 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:18.905148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:18.905932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:18.906130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:18.906409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:18.907871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:18.907976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:18.909317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:18.922436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 241/492] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:18.922436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:18.975637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a30d8-ac24-426f-afe8-ef1676e208dd None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.009334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.016437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.017147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.017320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.017599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.018510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:19.018612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.019982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.024353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441ebcb-1cb3-4fe9-93c2-0a722991a1b2 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.038890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 252/493] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:19.038890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.060694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0a2b5-d13b-4cdd-932d-77435ef5808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.061352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a2b5-d13b-4cdd-932d-77435ef5808e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.061595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a2b5-d13b-4cdd-932d-77435ef5808e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.061867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a2b5-d13b-4cdd-932d-77435ef5808e None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.062535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0a2b5-d13b-4cdd-932d-77435ef5808e None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:19.062830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0a2b5-d13b-4cdd-932d-77435ef5808e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.064255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0a2b5-d13b-4cdd-932d-77435ef5808e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.066005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.073808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 242/494] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:19.078576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f48c-9c0f-4a4e-b7ce-8bf7c023d4d8 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.078576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f48c-9c0f-4a4e-b7ce-8bf7c023d4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.078576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f48c-9c0f-4a4e-b7ce-8bf7c023d4d8 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.078576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 243/495] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:19.078576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1f48c-9c0f-4a4e-b7ce-8bf7c023d4d8 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.151718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6ba48-d4fc-4d0a-aca1-7eb4627642f4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.186895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.187524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.187798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.188091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.195969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1f48c-9c0f-4a4e-b7ce-8bf7c023d4d8 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.198299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.199764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:19.214429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:19.215315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:19.216180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:19.216180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:19.216180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:19.216180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 253/496] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:19.216393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.234556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170e993-1266-45ae-9f87-d7920f594ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.235221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e993-1266-45ae-9f87-d7920f594ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.235506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e993-1266-45ae-9f87-d7920f594ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.235780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e993-1266-45ae-9f87-d7920f594ac0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.243776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170e993-1266-45ae-9f87-d7920f594ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.245274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170e993-1266-45ae-9f87-d7920f594ac0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-a170e993-1266-45ae-9f87-d7920f594ac0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:19.258442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:19.259359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:19.260261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 244/497] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:19.260810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ae35c-2b27-4e05-b2b1-f2176f11ebca None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.261069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ae35c-2b27-4e05-b2b1-f2176f11ebca None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.261901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ae35c-2b27-4e05-b2b1-f2176f11ebca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.265045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.279570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-017ae35c-2b27-4e05-b2b1-f2176f11ebca None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:19.280876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-017ae35c-2b27-4e05-b2b1-f2176f11ebca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:19.323435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e9bf5-c38d-45d8-b606-af8120b09003 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.331796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.349169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 245/498] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:19.350477 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8272e-729d-45a8-a1ae-3d53044502fd None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.350774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8272e-729d-45a8-a1ae-3d53044502fd None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.351053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8272e-729d-45a8-a1ae-3d53044502fd None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.352608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 246/499] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:19.353324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8272e-729d-45a8-a1ae-3d53044502fd None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.361813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.362518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.362859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.363192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.374189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.375660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:19.390369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:19.391936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:19.392834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:19.392834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:19.392834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:19.392834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 254/500] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:19.393052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.475807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8272e-729d-45a8-a1ae-3d53044502fd None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.488778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.496062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8beb06-9ec7-4559-b5a0-10d311c5b413 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.509554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab667fcf-6fba-4a49-a83d-72d2afef05b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.510172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab667fcf-6fba-4a49-a83d-72d2afef05b9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.510446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab667fcf-6fba-4a49-a83d-72d2afef05b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.510723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab667fcf-6fba-4a49-a83d-72d2afef05b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.511359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab667fcf-6fba-4a49-a83d-72d2afef05b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:19.511664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab667fcf-6fba-4a49-a83d-72d2afef05b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.513065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab667fcf-6fba-4a49-a83d-72d2afef05b9 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.524492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 247/501] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:19.525653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98ea2-49fc-40cf-a5b9-d5615345c07b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.525940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98ea2-49fc-40cf-a5b9-d5615345c07b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.526218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98ea2-49fc-40cf-a5b9-d5615345c07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.531018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.531625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.531935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.532241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.532965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:19.533270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.534704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.543156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b5a98ea2-49fc-40cf-a5b9-d5615345c07b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:19.544372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b5a98ea2-49fc-40cf-a5b9-d5615345c07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:19.571118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 255/502] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:19.571857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.586733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.611740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 248/503] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:19.612751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a98ea2-49fc-40cf-a5b9-d5615345c07b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.671395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb25e0-b408-43c5-9233-bc99ca802937 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.706444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.707075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.707355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.707629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.709434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a98ea2-49fc-40cf-a5b9-d5615345c07b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.717645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:19.722344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:19.723352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:19.724361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 256/504] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:19.725296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.739117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.748412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.749035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.749317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.749591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.750236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:19.751034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.752433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.763458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 249/505] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:19.764275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.799302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.824440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61889e7-58af-4d42-93cd-fdb0b5071689 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.859005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.859633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.860433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.860713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.865181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78639721-65da-4ba4-bac3-8764cb1e72ab None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:19.867800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.869290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:19.882495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:19.883559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 257/506] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:19.889227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:19.899598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:19.900253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:19.900538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:19.900808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:19.956088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:19.969927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:19.971386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:19.986877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860650-cd2f-44c9-b977-663a41c775ea None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.006262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.021276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.021897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.022175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.022447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.033186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.034632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:20.049526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:20.050389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:20.051195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:20.051195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:20.051195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:20.051195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 258/507] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:20.051397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.078333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 250/508] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:20.078333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.153738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306aa544-2917-43a5-89f0-0874604587cc None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.173120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128ea7d-75f6-4c99-8ba0-eb7f85a01ae5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.176845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.188977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.189578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.189855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.190396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.191059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:20.191360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.192767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.200974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 259/509] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:20.201753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.211093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.211724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.212003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.212298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.214973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.220613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.222212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:20.233158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:20.234212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:20.235719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 251/510] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:20.236490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.302181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b281069-40aa-4d56-8478-343909be76e9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.334439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bb9d2-d486-4cdb-bb55-b0d94e243119 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.335644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.336311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.336593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.337596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:20.337898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.339333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.361548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 260/511] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:20.361548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.368260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.368879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.369158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.369427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.370066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:20.370369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.371723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.383134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 252/512] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:20.383922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.404776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.443625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.466249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f9ad3-5ef3-402e-aa08-c15dc6e9cecd None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.489718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedaea-b210-4cd5-8123-d5db9b2d1916 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.502300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.503086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.503364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.503722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.514193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.515607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.528050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.528711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.528992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.529266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:20.529941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:20.530998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:20.532611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:20.532611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:20.532611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:20.532611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 261/513] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:20.532611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.543787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.545336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:20.560084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:20.560978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:20.561787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:20.561787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:20.561787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:20.561787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 253/514] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:20.561988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.632796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.639579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6c9c4-cdc8-433e-ad50-e5c28caff5bd None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.660082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef645-d23f-4b9f-bec8-6a3b9a7893e2 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.669111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.674261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.674867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.675145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.675419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.676140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:20.676473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.677915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.694245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.694845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.694936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 262/515] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:20.695121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.695387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.695667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.762995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.764479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.798580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b460c26-143c-4e09-93f7-dcd7eb5549a5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.834521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:20.835117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:20.835313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:20.835583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:20.836256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:20.837689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:20.857876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 263/516] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:42:20 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:20.858665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.876905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.881633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 254/517] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:20.882796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:20.896190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:20.990334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd395d7-d52a-458a-a2b8-7a075267e53e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:20.999738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d344df-8615-450f-8550-fd87e9665ff7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.033794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.034417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.034699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.035039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.035600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:21.035903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.037304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.046752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.047416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.047700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.047979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.048668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:21.048973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.050354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.051575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 264/518] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:21.057649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.066206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 255/519] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:21.067134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.132312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.135041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.167552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6269471-bdfd-445e-9bba-8639e113f211 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.170040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214649f3-2d09-4c46-a742-7a2b949778c1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.204451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.205211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.205559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.205895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.207383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.208089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.208492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.208871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.220783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.222248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:21.238002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:21.239104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:21.239942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:21.239942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:21.239942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:21.239942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 256/520] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:21.240250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.287417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.289012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.351102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25878f51-5822-409c-87ce-dbd707f40fd6 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.361442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.372990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.389979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.390525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.390799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.391120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.398554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.399033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 265/521] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:21.399905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.404530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:21.423748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:21.424676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:21.425480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:21.426762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 257/522] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:21.427780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.515581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d358cdf-d247-4b44-9181-9bdb8bf829aa None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.543680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6731137-f6a4-47ba-8a9b-a029885f9f95 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.553323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.554166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.554527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.554918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.562848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.564478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:21.577035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:21.577953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 266/523] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:21.579408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.580887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.581177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.581416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.581677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.582319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:21.582616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.583978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.601805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 258/524] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:21.601805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.621829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.645921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.694286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b271565-d68d-4a11-a86b-7bccbd8c1d8d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.708688 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356d7a6-e4dc-407b-9983-87ed364834c4 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.729493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.730114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.730396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.730666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.741102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.742534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.743391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.743994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.744334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.744612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.745297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:21.745595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.746990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:21.757852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:21.758888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:21.759825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:21.759825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:21.759825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:21.765002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 267/525] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:21.765002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.772433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 259/526] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:21.773678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.860501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.880690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8144b-98e4-4a68-b40c-20c2392d7778 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.888837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:21.892926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095255c-2e6c-418d-b4a8-6f6853a82b63 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:21.918710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.919232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.919508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.919777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.920645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:21.920945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.922518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.934174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:21.934803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:21.935076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:21.935342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:21.946175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 268/527] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:21.946175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:21.947771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:21.949322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:21.962058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:21.962881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:21.963791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 260/528] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:21.964271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.058156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec470fe4-efff-4a82-9dda-ff90dbbe82b1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.068055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c82b13-5b43-4375-b6fd-39593887daf1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.095569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.096549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.097090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.097369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.098010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:22.098641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.099676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.102073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.102683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.102956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.103225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.103969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=86456c6dbe4e454e9f255960b70c0832) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:22.104318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.105731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.109869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:22.120176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 261/529] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/role_assignments?user.id=86456c6dbe4e454e9f255960b70c0832&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:22.120176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.124329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 269/530] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:22.125240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.148574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:22.251971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdf1fa-3db4-4616-8b40-c467abd236b3 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.281664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ac12-d687-4ef3-9e7f-5842d43a344c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.289270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.289884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.290172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.290432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.323612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.324285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.324566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.324942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.325638 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:22.326190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.327533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.347297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 270/531] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:22.348278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.367574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.369431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.396240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:22.435396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:22.481141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 262/532] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:22.482411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.485525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb690-536b-44ab-915d-65809330f0c1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.528650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.529230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.529804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.530063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.537975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.539448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:22.553054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:22.553939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:22.554776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 271/533] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:22.555675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.593488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff1bcf-0a6d-436f-a06f-ee93d230203e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.619419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:22.631657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.632636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.633153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.633662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.634622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=770e3b2a34d74195bcdf1d0b6f696cda) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:22.635121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.636783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.643141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 263/534] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/role_assignments?user.id=770e3b2a34d74195bcdf1d0b6f696cda&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:22.644244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.691255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f796d-a9d1-47db-94fd-4855e89a9d56 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.729610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:22.756713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.757794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.758271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.759639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.762681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fc564-46fd-4c4e-88f7-fdb1e217660e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.785663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.787583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:22.821933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:22.822863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:22.823684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:22.823684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:22.823684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:22.828667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 272/535] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:22.834925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.849582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:22.850556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:22.851037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:22.851562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:22.852592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:22.853101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:22.854776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:22.869969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 264/536] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:22.875545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:22.931308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:22.987670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239c770-c51e-45fc-bf1f-20367877d5ca None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:22.990052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45180725-5acb-49cd-8d7c-a5c07a7ca5e5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:23.030079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:23.030889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.031249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.031601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.032580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcdbcb-931a-4bb2-a18d-117516dc6cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:23.032958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:23.033328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:23.033820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcdbcb-931a-4bb2-a18d-117516dc6cdf None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.034265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcdbcb-931a-4bb2-a18d-117516dc6cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.034701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcdbcb-931a-4bb2-a18d-117516dc6cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.034962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:23.035818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcdbcb-931a-4bb2-a18d-117516dc6cdf None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=88c76a4a9747469d88a0a1a5748452a1) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:23.036367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcdbcb-931a-4bb2-a18d-117516dc6cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:23.038722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcdbcb-931a-4bb2-a18d-117516dc6cdf None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:23.046582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 265/537] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/role_assignments?user.id=88c76a4a9747469d88a0a1a5748452a1&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:23.048070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c34c48-8baf-453b-b04c-bd20bd066f8b None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.048531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c34c48-8baf-453b-b04c-bd20bd066f8b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.048979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c34c48-8baf-453b-b04c-bd20bd066f8b None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.050639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 266/538] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:23.060171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 273/539] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:23.060701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3fc53-b2df-4a6c-9adc-c9d4b012f569 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.060701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3fc53-b2df-4a6c-9adc-c9d4b012f569 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.060701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3fc53-b2df-4a6c-9adc-c9d4b012f569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.065619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:23.097284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-aad3fc53-b2df-4a6c-9adc-c9d4b012f569 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:23.098527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aad3fc53-b2df-4a6c-9adc-c9d4b012f569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:23.104440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:23.191685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd68672-eeb7-4e6d-af8e-f19cb9ed84d4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:23.237049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:23.238605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.238990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.239390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.253020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:23.255019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:23.276482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 267/540] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:23.281609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:23.282466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:23.283662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 274/541] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:23.288667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3fc53-b2df-4a6c-9adc-c9d4b012f569 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:23.309739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:23.356520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:23.474528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3fc53-b2df-4a6c-9adc-c9d4b012f569 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:23.494188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:23.509628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377c148-b068-415b-8dfa-bdd73b7110ff None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:23.545882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:23.546888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.547403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.547870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.548797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:23.549304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:23.550963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:23.573187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 268/542] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:23.577535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:23.578492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.580069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:23.580966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.581945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.603649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:23.610721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:23.611618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:23.612395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:23.619038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 275/543] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:23.626601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:23.738497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a766ec-0296-43c6-86b9-9d85e54c6052 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:23.741670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:23.795662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43301860-31d4-48d2-8869-b5824783ab9e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:23.832165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:23.833582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.834523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.836294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.837231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=256af8436dd842fdb4ebe90ec9be26ae) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:23.838049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:23.838633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:23.839428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:23.841103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:23.841581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:23.842065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:23.842928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:23.844040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:23.846841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:23.856325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 269/544] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:23 2022] GET /identity/v3/role_assignments?user.id=256af8436dd842fdb4ebe90ec9be26ae&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:23.861326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 276/545] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:23.866856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:24.003374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31033eb7-ab4b-43e5-991b-984bd79757bb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:24.044648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:24.045316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:24.045599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:24.045869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:24.061963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:24.063647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:24.079920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:24.080878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:24.082364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:24.082364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:24.082364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:24.082364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 270/546] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:24.086875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:24.129674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:24.238571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91395e35-6735-46c0-b942-e9273595d319 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:24.280983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07478a-0bb2-44f9-a7fd-d7b47e0ccd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:24.281595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07478a-0bb2-44f9-a7fd-d7b47e0ccd2b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:24.282390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07478a-0bb2-44f9-a7fd-d7b47e0ccd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:24.282662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07478a-0bb2-44f9-a7fd-d7b47e0ccd2b None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:24.283299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07478a-0bb2-44f9-a7fd-d7b47e0ccd2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:24.283593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07478a-0bb2-44f9-a7fd-d7b47e0ccd2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:24.284979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07478a-0bb2-44f9-a7fd-d7b47e0ccd2b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:24.301861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 277/547] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:24.306935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:24.345605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:24.437939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68bfda-d8ff-406e-b71b-11227082aea7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:24.487997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:24.488691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:24.488974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:24.489249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:24.498186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:24.499280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3089-eade-4a05-8504-12948a78a873 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:24.499663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:24.500014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3089-eade-4a05-8504-12948a78a873 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:24.500731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3089-eade-4a05-8504-12948a78a873 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:24.502720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 278/548] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:24.508354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cbcf-2aae-48a7-9459-b73d979d4a00 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:24.508946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cbcf-2aae-48a7-9459-b73d979d4a00 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:24.509416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cbcf-2aae-48a7-9459-b73d979d4a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:24.517486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:24.519360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:24.520525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 271/549] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:24.527059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:24.553846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a864cbcf-2aae-48a7-9459-b73d979d4a00 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:24.558360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a864cbcf-2aae-48a7-9459-b73d979d4a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:24.637206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:24.660415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcab78-010d-412c-a487-46a5526bb6f0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:24.699195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:24.699823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:24.700112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:24.700469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:24.701214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:24.701535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:24.702998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:24.725754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 279/550] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:24.742100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 272/551] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 10 16:42:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:24.747675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864cbcf-2aae-48a7-9459-b73d979d4a00 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:24.754308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:24.756120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:24.931480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d052eb7-9e5e-4f31-b97c-71830cb441fd None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:24.952871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864cbcf-2aae-48a7-9459-b73d979d4a00 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:24.975647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:24.976319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:24.976602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:24.976948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:24.981353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:24.997728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:25.004327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:25.005303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:25.006269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 273/552] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:25.018159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.032750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.033373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.033657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.033929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.156847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:25.158426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:25.162266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:25.186706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301e8a5-1ab2-4edd-a2f6-90c3febba815 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:25.236090 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61ded9-8910-4752-974f-66a834d81e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.236736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61ded9-8910-4752-974f-66a834d81e41 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.237036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61ded9-8910-4752-974f-66a834d81e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.237335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61ded9-8910-4752-974f-66a834d81e41 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.238521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61ded9-8910-4752-974f-66a834d81e41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:25.238828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61ded9-8910-4752-974f-66a834d81e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:25.240498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61ded9-8910-4752-974f-66a834d81e41 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:25.256196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 274/553] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:25.256196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99665-21e5-447b-a8d3-56d7c1fddace None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.256196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99665-21e5-447b-a8d3-56d7c1fddace None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.256196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99665-21e5-447b-a8d3-56d7c1fddace None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.259437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 275/554] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:25.260720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1988aa-f76d-462a-bbee-898a263a01f9 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.261004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1988aa-f76d-462a-bbee-898a263a01f9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.261283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1988aa-f76d-462a-bbee-898a263a01f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.283352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 280/555] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:25.284329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-db1988aa-f76d-462a-bbee-898a263a01f9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:25.284515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.285525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-db1988aa-f76d-462a-bbee-898a263a01f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:25.362071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:25.382247 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 276/556] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:25.383262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1988aa-f76d-462a-bbee-898a263a01f9 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.409404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4085-6a24-484b-bbad-7ef72ac8058e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:25.415056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:25.453263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.454146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.454588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.455104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.466285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:25.468013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:25.484602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:25.486176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:25.487038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:25.487038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:25.487038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:25.492322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 281/557] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:25.493309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.513505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1988aa-f76d-462a-bbee-898a263a01f9 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:25.562434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.563352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.563637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.563909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.574968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:25.576489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:25.591230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:25.598020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:25.598861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:25.598861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:25.598861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:25.598861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 277/558] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:25.598861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.610955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4525a-5a26-48bf-963b-2df2b561125b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:25.643245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:25.646846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6d3e3-3915-43f1-9ee7-2cef5901e86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.647492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6d3e3-3915-43f1-9ee7-2cef5901e86f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.647775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6d3e3-3915-43f1-9ee7-2cef5901e86f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.648049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6d3e3-3915-43f1-9ee7-2cef5901e86f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.659779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6d3e3-3915-43f1-9ee7-2cef5901e86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-e3d6d3e3-3915-43f1-9ee7-2cef5901e86f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:25.664938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:25.665825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:25.666723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 282/559] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:25.667382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b2e2d-01e8-4cc7-a9b8-02f0bcd49d77 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.667932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b2e2d-01e8-4cc7-a9b8-02f0bcd49d77 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.667932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b2e2d-01e8-4cc7-a9b8-02f0bcd49d77 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.672214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 283/560] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:25.672214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b2e2d-01e8-4cc7-a9b8-02f0bcd49d77 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.690050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:25.714786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78974dff-cd09-46d6-ba17-78c6b0ffbd8c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:25.754144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.754810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.755087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.755357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.756023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:25.756365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:25.757720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:25.769731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 278/561] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:25.770564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.798230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b2e2d-01e8-4cc7-a9b8-02f0bcd49d77 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:25.833617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e461e-8049-4193-a5be-4c2207599e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.834211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e461e-8049-4193-a5be-4c2207599e13 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.834490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e461e-8049-4193-a5be-4c2207599e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.834760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e461e-8049-4193-a5be-4c2207599e13 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.835389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e461e-8049-4193-a5be-4c2207599e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:25.835691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e461e-8049-4193-a5be-4c2207599e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:25.837089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e461e-8049-4193-a5be-4c2207599e13 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:25.849328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 284/562] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:25.850566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e932cc9-0baa-4710-8292-63fee71afd70 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.850851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e932cc9-0baa-4710-8292-63fee71afd70 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.851126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e932cc9-0baa-4710-8292-63fee71afd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.869154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9e932cc9-0baa-4710-8292-63fee71afd70 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:25.870344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9e932cc9-0baa-4710-8292-63fee71afd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:25.878935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:25.883494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9bb73-ca36-429a-a03a-244acb3eca27 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:25.922549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:25.923743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:25.924649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:25.925250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:25.926208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:25.926715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:25.928350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:25.937420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 279/563] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:25.938467 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.944088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 285/564] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:25.945228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e932cc9-0baa-4710-8292-63fee71afd70 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:25.954759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.052795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e932cc9-0baa-4710-8292-63fee71afd70 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.057804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5006e23-0fa1-47c3-857b-412ea15dd47d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.086459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.087218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.087504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.087837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.092572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.093190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.093468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.093738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.095212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.096687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.101604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.103005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:26.107715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:26.108668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:26.109518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 286/565] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:26.110028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:26.115589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:26.116702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:26.117853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:26.121183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 280/566] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:42:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:26.122034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.146372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.179143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.214606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a23c33-397e-4039-9143-e91283250ac3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.223183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db314e6-059c-430a-8c95-612e2e630b47 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.249389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.250012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.250293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.251056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.255997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71acbf2-a12c-4353-8ae7-382b11cf2be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.256647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71acbf2-a12c-4353-8ae7-382b11cf2be9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.256925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71acbf2-a12c-4353-8ae7-382b11cf2be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.257195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71acbf2-a12c-4353-8ae7-382b11cf2be9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.260858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.262568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.267978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71acbf2-a12c-4353-8ae7-382b11cf2be9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-b71acbf2-a12c-4353-8ae7-382b11cf2be9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:26.273048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:26.274720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 281/567] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:26.276427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2435-4d57-4550-b811-5017fe7ea92d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.277174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2435-4d57-4550-b811-5017fe7ea92d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.277174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2435-4d57-4550-b811-5017fe7ea92d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.277628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 282/568] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:26.277731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:26.279334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:26.281259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:26.281259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:26.281259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:26.282482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa2435-4d57-4550-b811-5017fe7ea92d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.283028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 287/569] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:26.283870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.361878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.381957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa2435-4d57-4550-b811-5017fe7ea92d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.382543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6bf00-9c6e-41ed-a2b1-ae1440caf597 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.399413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.416135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.416648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.416760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.417026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.417263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.417360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.417545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.417818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.417997 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:26.418300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.418505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:26.418841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.419725 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.420383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.434540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 283/570] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:26.435207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.446672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 288/571] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:26.446672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.534475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5d75-a948-4296-b752-977bf91e533e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.543231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acef53-4bdd-4f4a-b4be-951e1ff63c53 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.569217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece04679-3caf-4581-9605-38163ebf6a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.569824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece04679-3caf-4581-9605-38163ebf6a1c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.570100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece04679-3caf-4581-9605-38163ebf6a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.570369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece04679-3caf-4581-9605-38163ebf6a1c None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.571021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece04679-3caf-4581-9605-38163ebf6a1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:26.571319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece04679-3caf-4581-9605-38163ebf6a1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.572715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece04679-3caf-4581-9605-38163ebf6a1c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.578271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.579418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.579695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.579989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.580667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:26.580965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 284/572] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:26.581090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.582097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b36da0-a674-4bad-b7d2-c16c35406b75 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.582375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b36da0-a674-4bad-b7d2-c16c35406b75 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.582639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b36da0-a674-4bad-b7d2-c16c35406b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.585808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.589392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.597126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 289/573] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:26.597932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.601410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-94b36da0-a674-4bad-b7d2-c16c35406b75 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:26.602594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-94b36da0-a674-4bad-b7d2-c16c35406b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:26.633943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.669846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 285/574] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:26.670966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b36da0-a674-4bad-b7d2-c16c35406b75 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.706826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955543-6623-4b3e-a414-78d100037719 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.740799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.741405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.741682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.741948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.742778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=f64c104787604ea4b22e2af2507b3f0a) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:26.743088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.744558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.749810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 290/575] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/role_assignments?user.id=f64c104787604ea4b22e2af2507b3f0a&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:26.751136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.800381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b36da0-a674-4bad-b7d2-c16c35406b75 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.838936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.839583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.839862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.840133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.842743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.852596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.854077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.860778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:26.865259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc36c4-7e1d-44f6-907a-bd7da97bdb55 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:26.870987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:26.871997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:26.872931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:26.872931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:26.872931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:26.877559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 286/576] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:26.878321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:26.904743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:26.905493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:26.905767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:26.906037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:26.917691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:26.919157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:26.937395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:26.939964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:26.941211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:26.941211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:26.941211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:26.941211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 291/577] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:26.941211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.004625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09724b2-957a-40e3-a092-1e08a2e70731 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.042509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.043267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.043551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.043936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.052805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.053074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ae485-8bd8-4972-91a2-1ca11e9fe052 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.054256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:27.066778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:27.067850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 287/578] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:27.069323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.097445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.098085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.098558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.098838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.112440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:27.117782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:27.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:27.120394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:27.121817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 292/579] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:27.122659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.138279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.177084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0fdf7-81b8-4132-afd6-c9efa09b337c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.212634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.213375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.213657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.213934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.214578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:27.214880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.216504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.228913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 288/580] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:27.229726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.238109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff1b4c-f577-4d50-8785-3f13cce79e53 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.273509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.274121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.274387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.274662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.275303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:27.275603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.276999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.289847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 293/581] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:27.290745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.351201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e19d4-1383-4e41-82d4-d7481f157770 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.360414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.376383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.389658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.390263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.390539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.390845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.392313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:27.392420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.393791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.410041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad339d-0d79-4636-8855-bd38f897b10d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.418027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 289/582] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:27.418027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.455216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.455856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.456228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.456510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.457176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:27.457480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.458872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.468174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 294/583] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:27.468174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.524045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa154c4-298a-40fb-9308-35b51939caca None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.561012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.561848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.562125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.562410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.570809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.572120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61b0c-125b-4499-9025-40a3fcba1b1b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.572323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:27.584096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:27.585020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:27.585890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 290/584] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:27.586368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.613376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.614337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.615301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.615616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.615888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.618498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.625425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.626795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:27.637853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:27.638759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:27.639828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 295/585] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:27.640430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.694753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7a796-d055-4a4a-88e4-4c82565ade55 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.730256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.730813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.731091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.731363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.742229 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.743675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:27.758443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:27.759365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:27.760261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:27.760261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:27.760261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:27.760261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 291/586] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:27.760490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.761803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2a4c8-2841-497a-8f2c-97356eb10298 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.800620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.801291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.801573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.801968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.814162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.815895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:27.832291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:27.833218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:27.834062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:27.834062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:27.834062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:27.834062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 296/587] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:27.834288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.849913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.877769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb94567-23c0-4466-bc00-96c159cf3517 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.886389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:27.913178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.913842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.914126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.914398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.915088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:27.915395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.916872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:27.934479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 292/588] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:27.934479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:27.948360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e588-1777-403a-b0a0-165dd8ade1aa None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:27.985503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:27.986114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:27.986399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:27.986668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:27.987381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:27.987682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:27.989118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.010973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 297/589] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:28.010973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.052762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b97d40-8aa1-484d-9fac-1b6ea1205c52 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.089993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.090601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.090879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.091150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.091784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:28.092085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.093600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.096538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:28.112041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 293/590] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:28.112843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.119628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3c5eb-2119-4419-83de-301d9caa4771 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.122233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:28.156503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.157691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.157968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.158231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.158872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:28.159169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.160566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.176497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 298/591] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:28.176497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.229192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea03844-ac64-4404-8a38-986cd9bb30d7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.267977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.268748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.269414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.269694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/86456c6dbe4e454e9f255960b70c0832/roles/6aaf096343b84cfea3eb6e8fd11ba645` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.286336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=86456c6dbe4e454e9f255960b70c0832, role_id=6aaf096343b84cfea3eb6e8fd11ba645)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.288134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.288594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7744f5-cebd-4dc1-af01-78348db5093c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.304876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 294/592] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:28 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/86456c6dbe4e454e9f255960b70c0832/roles/6aaf096343b84cfea3eb6e8fd11ba645 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:28.305975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.345764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.347056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.347409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.347796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.365936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.367456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.380538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:28.401237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:28.402274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:28.403137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:28.403137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:28.403137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:28.403838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 299/593] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:28.404593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.441034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:28.463693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2149cf3-aa9c-4bf1-8fe7-d7765f36a300 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.514108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.514758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.515188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.515468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.523956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.525413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:28.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:28.539333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:28.540242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 295/594] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:28.544851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.566754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5f6b6-d5c9-486b-90db-d726206b1247 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.608558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.609593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.609593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.609895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.618441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.619837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:28.632906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:28.633887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 300/595] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:28.634798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.673749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690487-0943-4278-a966-111569ffa1b9 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.683653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:28.725811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.726683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.727112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.727529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.728977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:28.729475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.731174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:28.732433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.750300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 296/596] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:28.751439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.770604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39746137-46b2-484a-9c53-44ec3250ae65 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.814956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.815946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.816424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.816808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.817689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:28.818098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.819650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.833328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 301/597] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:28.834375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.879119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bcb13-98f6-48e5-8537-2667da15d581 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.915936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a625b66-a0c1-44c2-b084-6cfab6996338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.916581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a625b66-a0c1-44c2-b084-6cfab6996338 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.916865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a625b66-a0c1-44c2-b084-6cfab6996338 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.917141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a625b66-a0c1-44c2-b084-6cfab6996338 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.917899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a625b66-a0c1-44c2-b084-6cfab6996338 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:28.918209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a625b66-a0c1-44c2-b084-6cfab6996338 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:28.919661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a625b66-a0c1-44c2-b084-6cfab6996338 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:28.937433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 297/598] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:28.938687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc64640-d996-469e-8a91-830e35a4e128 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.938976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc64640-d996-469e-8a91-830e35a4e128 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.939240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc64640-d996-469e-8a91-830e35a4e128 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.940869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 298/599] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:28.941389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc64640-d996-469e-8a91-830e35a4e128 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:28.953213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bca653-9ec1-4256-a165-809030ed23e9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:28.966750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:28.994762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:28.995395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:28.995733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:28.995919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:28.999242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:29.004796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.006828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:29.018645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:29.019538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 302/600] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:29.025778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.052949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc64640-d996-469e-8a91-830e35a4e128 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.089422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea97f8-4495-4e6a-9e80-58e2a3713dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.090014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea97f8-4495-4e6a-9e80-58e2a3713dc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.090294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea97f8-4495-4e6a-9e80-58e2a3713dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.090573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea97f8-4495-4e6a-9e80-58e2a3713dc0 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/770e3b2a34d74195bcdf1d0b6f696cda/roles/6aaf096343b84cfea3eb6e8fd11ba645` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.108062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea97f8-4495-4e6a-9e80-58e2a3713dc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=770e3b2a34d74195bcdf1d0b6f696cda, role_id=6aaf096343b84cfea3eb6e8fd11ba645)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.110317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea97f8-4495-4e6a-9e80-58e2a3713dc0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.132191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 299/601] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:28 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/770e3b2a34d74195bcdf1d0b6f696cda/roles/6aaf096343b84cfea3eb6e8fd11ba645 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:29.132191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcb49-91ec-48f0-9408-0f632bb6f3e5 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.132191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcb49-91ec-48f0-9408-0f632bb6f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.132191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcb49-91ec-48f0-9408-0f632bb6f3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.150079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-591dcb49-91ec-48f0-9408-0f632bb6f3e5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:29.151422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-591dcb49-91ec-48f0-9408-0f632bb6f3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:29.153927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951b3ad-8a06-4fc1-8e19-a4eb5a988dcf None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.192210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.192949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.193358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.193699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.206845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.208488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:29.229102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:29.230303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:29.231403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:29.231403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:29.231403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:29.237420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 303/602] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:29.238535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.245766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 300/603] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:29.247011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dcb49-91ec-48f0-9408-0f632bb6f3e5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.252607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:29.304781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:29.366531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8eeea-c8e3-418c-99ae-b33107423791 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.384333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dcb49-91ec-48f0-9408-0f632bb6f3e5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.404798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.405585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.405871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.406148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.406882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:29.407190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.408675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.422487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 304/604] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:29.428346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.438621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.439335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.444173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.444173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.444173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:29.444173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.444173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.464895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 301/605] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:29.465827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.538645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:29.555729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745c5c5-cdfd-4d94-9567-666ad75e7fb5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.567666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:29.580316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad0fe2-0850-442a-ace8-4828e0b7fb15 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.598328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.599204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.599630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.600046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.612437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:29.615672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.616614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.617056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:29.617242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:29.618307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:29.619316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:29.620357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/88c76a4a9747469d88a0a1a5748452a1/roles/6aaf096343b84cfea3eb6e8fd11ba645` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.623381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 305/606] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:29.624168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.637390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=88c76a4a9747469d88a0a1a5748452a1, role_id=6aaf096343b84cfea3eb6e8fd11ba645)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.639074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.656951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 302/607] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:29 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/88c76a4a9747469d88a0a1a5748452a1/roles/6aaf096343b84cfea3eb6e8fd11ba645 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:29.658199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.775361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a7f23-96c9-4310-8fe4-20ae74d75134 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.805031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e9254-508b-4dcd-b5f1-7f9a6502a6c7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:29.819938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.820617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.820904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.821179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.826240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:29.832599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.834451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.855296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:29.856136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:29.856454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:29.856738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:29.857877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:29.858420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:29.861889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:29.863174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:29.864430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:29.866259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:29.869942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 306/608] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:29.876121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.885635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 303/609] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:29.892597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:29.913672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:30.016001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1016ba3-349c-470c-a01b-51f6fa581870 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:30.039119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7576db1-8607-47f1-9100-7ba38641c0ba None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:30.076726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:30.077678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.078114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.078568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.079425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:30.079884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:30.083841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:30.105686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 307/610] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:30.113359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:30.120977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:30.121586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.121876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.122158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.149234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:30.150800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:30.181767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:30.182981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:30.184103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:30.184103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:30.184103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:30.192558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 304/611] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:30.193496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:30.213386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:30.229496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984202a2-d36a-49aa-bc0b-558e4faafd7d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:30.267770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:30.268831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.269337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.269818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/256af8436dd842fdb4ebe90ec9be26ae/roles/6aaf096343b84cfea3eb6e8fd11ba645` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.285168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:30.288125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=256af8436dd842fdb4ebe90ec9be26ae, role_id=6aaf096343b84cfea3eb6e8fd11ba645)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:30.289932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:30.311686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 308/612] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:30 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/256af8436dd842fdb4ebe90ec9be26ae/roles/6aaf096343b84cfea3eb6e8fd11ba645 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:30.418692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b9895-399b-4314-ba49-db4ed075b2bb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:30.462427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8693d-ba42-4140-b846-ed8df76c361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:30.463068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8693d-ba42-4140-b846-ed8df76c361e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.463352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8693d-ba42-4140-b846-ed8df76c361e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.463627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8693d-ba42-4140-b846-ed8df76c361e None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.464321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8693d-ba42-4140-b846-ed8df76c361e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:30.464630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8693d-ba42-4140-b846-ed8df76c361e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:30.466103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8693d-ba42-4140-b846-ed8df76c361e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:30.487562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 305/613] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:30.489052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:30.601932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94c99d-7f9d-4b05-868e-fe289f90fa13 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.602249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94c99d-7f9d-4b05-868e-fe289f90fa13 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.602828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94c99d-7f9d-4b05-868e-fe289f90fa13 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.606516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 306/614] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:30.609990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79251e68-7c34-4a54-ade9-3b3dd4e27cb5 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.610288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79251e68-7c34-4a54-ade9-3b3dd4e27cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.610591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79251e68-7c34-4a54-ade9-3b3dd4e27cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.628832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6396a03-b05a-4972-8432-9e63a44eb1da None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:30.638927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-79251e68-7c34-4a54-ade9-3b3dd4e27cb5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:30.640935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-79251e68-7c34-4a54-ade9-3b3dd4e27cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:30.669218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:30.669780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.670060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.670332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.679649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:30.681080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:30.683044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:30.696364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:30.697591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:30.698760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:30.706204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 309/615] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:30.707825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:30.757497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 307/616] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:30.805323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79251e68-7c34-4a54-ade9-3b3dd4e27cb5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:30.841200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42ff04-617d-48e9-9e42-2c72c25c6e66 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:30.856506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:30.880184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:30.880807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:30.881088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:30.881375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:30.882273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:30.882584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:30.884388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:30.912547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 310/617] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 10 16:42:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:30.913981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:30.981166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:31.031115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91030bad-5f30-400e-8620-7ad4aa1e5d8f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.054282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79251e68-7c34-4a54-ade9-3b3dd4e27cb5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.075058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.075707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.075987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.076290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.091811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:31.096294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:31.098576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.099450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.100170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.101097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.126820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:31.132692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:31.133616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:31.134482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:31.134482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:31.134482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:31.136717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 311/618] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:31.137061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:31.137948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:31.138794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 308/619] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:31.147060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.147209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.282011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae2d4f-a4e7-4178-a8a4-11d7a18b97b4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.300482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:31.321994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:31.349710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02d845-0f77-40f2-8fe5-f73e4424862b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.368859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.369483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.369764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.370039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.370693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:31.370993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:31.372392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:31.387044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.387814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.388220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.388593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.389362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:31.389769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:31.391267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:31.398595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 312/620] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:31.400172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 309/621] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:31.413245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.413374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.553607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f9a58-631a-428a-a2f9-e7d2f6c479b0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.566450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd65523-e86a-4b80-a361-ed8361c95017 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.590277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb47be-ecaa-4078-9384-12789e151fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.591169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb47be-ecaa-4078-9384-12789e151fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.591598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb47be-ecaa-4078-9384-12789e151fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.592028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb47be-ecaa-4078-9384-12789e151fd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.599799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb47be-ecaa-4078-9384-12789e151fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:31.601504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb47be-ecaa-4078-9384-12789e151fd8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:31.604585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:31.612640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ec7f7-3891-4e8d-bc89-af2a5cc8d3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.613424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ec7f7-3891-4e8d-bc89-af2a5cc8d3fa None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.613812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ec7f7-3891-4e8d-bc89-af2a5cc8d3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.614196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ec7f7-3891-4e8d-bc89-af2a5cc8d3fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-2cbb47be-ecaa-4078-9384-12789e151fd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:31.616240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:31.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:31.618267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:31.624721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 310/622] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:31.626393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec101658-2130-46ba-8a18-d5f48839e634 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.626827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec101658-2130-46ba-8a18-d5f48839e634 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.627253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec101658-2130-46ba-8a18-d5f48839e634 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.628834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ec7f7-3891-4e8d-bc89-af2a5cc8d3fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:31.629739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 311/623] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:31.630630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec101658-2130-46ba-8a18-d5f48839e634 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.634779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ec7f7-3891-4e8d-bc89-af2a5cc8d3fa None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-4d4ec7f7-3891-4e8d-bc89-af2a5cc8d3fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:31.656228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:31.657939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:31.659025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:31.659025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:31.659025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:31.659025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 313/624] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:31.660908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b70e3a-bb2d-4af0-8c84-c6546f395afb None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.661308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b70e3a-bb2d-4af0-8c84-c6546f395afb None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.661591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b70e3a-bb2d-4af0-8c84-c6546f395afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.680901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-64b70e3a-bb2d-4af0-8c84-c6546f395afb None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:31.682266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-64b70e3a-bb2d-4af0-8c84-c6546f395afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:31.712544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:31.748685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec101658-2130-46ba-8a18-d5f48839e634 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.764582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 314/625] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:31.765750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b70e3a-bb2d-4af0-8c84-c6546f395afb None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.785695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.786598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.787053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.787496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.788366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:31.788827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:31.790491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:31.808344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 312/626] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:31.809431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.876770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:31.898514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b70e3a-bb2d-4af0-8c84-c6546f395afb None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.922631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984eb548-af9d-468d-ad80-7adf424ea61b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:31.938102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.939219 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.939830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.940365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.941516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:31.942043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:31.943652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:31.957747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 315/627] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:31.958849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:31.964292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:31.964950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:31.965234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:31.965532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:31.978451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:31.983911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:31.984930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:31.985738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:31.990124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 313/628] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:31.991044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.016863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.075301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d58560-e6e7-4d01-bf72-036ad1ecbeb1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.099264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e516e1-69b0-4e3e-992b-5dc8ab1b472d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.109960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.110683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.111026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.111364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/f64c104787604ea4b22e2af2507b3f0a/roles/6aaf096343b84cfea3eb6e8fd11ba645` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.121782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.126262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=f64c104787604ea4b22e2af2507b3f0a, role_id=6aaf096343b84cfea3eb6e8fd11ba645)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:32.127796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:32.137738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a369868-0335-482d-994b-b34ad541d937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.138576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a369868-0335-482d-994b-b34ad541d937 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.139109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a369868-0335-482d-994b-b34ad541d937 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.139558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a369868-0335-482d-994b-b34ad541d937 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.141603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 316/629] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:31 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/f64c104787604ea4b22e2af2507b3f0a/roles/6aaf096343b84cfea3eb6e8fd11ba645 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:32.142555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.148693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a369868-0335-482d-994b-b34ad541d937 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:32.150178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a369868-0335-482d-994b-b34ad541d937 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-1a369868-0335-482d-994b-b34ad541d937 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:32.165908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:32.166940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:32.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 314/630] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:42:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:32.168500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3e073-cee8-4e40-ac7a-97f15d1ede58 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.168590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3e073-cee8-4e40-ac7a-97f15d1ede58 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.169103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3e073-cee8-4e40-ac7a-97f15d1ede58 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.174313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 315/631] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:32.177019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9dd92-7fb4-421b-a0c2-280b308656e3 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.177306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9dd92-7fb4-421b-a0c2-280b308656e3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.177580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9dd92-7fb4-421b-a0c2-280b308656e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.197525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-59c9dd92-7fb4-421b-a0c2-280b308656e3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:32.198855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-59c9dd92-7fb4-421b-a0c2-280b308656e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:32.251512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.291161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 316/632] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:32.292252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9dd92-7fb4-421b-a0c2-280b308656e3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.308952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e24e3-b6c0-41f8-8a89-548cbc466b30 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.364930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.366592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.367560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.368054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.369236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:32.370191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:32.371610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:32.382252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 317/633] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:32.383345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.430147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9dd92-7fb4-421b-a0c2-280b308656e3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.443347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.469394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.470282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.470708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.471125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.472003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:32.472499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:32.474072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:32.493109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 317/634] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:32.493983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.521659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1f59a-441f-4c5c-9a59-db48048c8f06 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.551405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.566846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.567783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.568257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.568692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.579884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:32.585433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:32.586480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:32.587462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:32.589693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 318/635] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:32.590618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.615307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5ae52-9798-4977-b96b-45c0e08300df None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.654127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.655195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.655584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.655968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.667956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:32.669620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:32.685421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:32.686529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:32.687594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:32.687594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:32.687594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:32.689524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 318/636] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:32.690280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.721104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.726462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d60ae4-21a4-4f8e-90a6-57ff822c05cb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.765213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.765923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.766236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.766514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.807613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badc484-dcb2-47bf-aa27-df97066ab8ee None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:32.821077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.843088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:32.846946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:32.847741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:32.848243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:32.848475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:32.848830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:32.850276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:32.850918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:32.852981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:32.865412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 319/637] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:32.866552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.957935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:32.963600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 319/638] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:32.964490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:32.979642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f90c-f42b-44fd-b812-315ea830375a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.021081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4000746-3d2c-46b7-8de2-a3e1746cd62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.021920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000746-3d2c-46b7-8de2-a3e1746cd62f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.022721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000746-3d2c-46b7-8de2-a3e1746cd62f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.023140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000746-3d2c-46b7-8de2-a3e1746cd62f None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.024064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4000746-3d2c-46b7-8de2-a3e1746cd62f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:33.024554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4000746-3d2c-46b7-8de2-a3e1746cd62f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.026046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4000746-3d2c-46b7-8de2-a3e1746cd62f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.043633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 320/639] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:33.045177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f96e5e-c3ed-4e55-af30-6dd50d97fb42 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.045461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f96e5e-c3ed-4e55-af30-6dd50d97fb42 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.045737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f96e5e-c3ed-4e55-af30-6dd50d97fb42 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.048201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 321/640] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:33.048616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f96e5e-c3ed-4e55-af30-6dd50d97fb42 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.058324 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:33.079448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7dbf-d9c0-4d1f-b228-4b242bad18cc None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.115361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.116065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.116297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.116567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.129453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.131691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:33.150627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:33.151802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:33.152792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:33.152792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:33.152792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:33.152792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 320/641] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:33.153864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.163935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f96e5e-c3ed-4e55-af30-6dd50d97fb42 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.201475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78fa92-4092-4b79-a8f9-77adb6441d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.202109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78fa92-4092-4b79-a8f9-77adb6441d1d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.202396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78fa92-4092-4b79-a8f9-77adb6441d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.202672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78fa92-4092-4b79-a8f9-77adb6441d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.212578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78fa92-4092-4b79-a8f9-77adb6441d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.214019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78fa92-4092-4b79-a8f9-77adb6441d1d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.215723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-4d78fa92-4092-4b79-a8f9-77adb6441d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:33.229938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:33.230955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:33.231813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:33.231813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:33.231813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:33.231813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 322/642] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:33.232438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d4e4-1ff3-4520-a603-0c6df9e0a69d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.232516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d4e4-1ff3-4520-a603-0c6df9e0a69d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.232735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d4e4-1ff3-4520-a603-0c6df9e0a69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.251898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-af60d4e4-1ff3-4520-a603-0c6df9e0a69d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:33.253552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-af60d4e4-1ff3-4520-a603-0c6df9e0a69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:33.267228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e815f-c313-4c4e-9462-5aa1247bfbe4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.303128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.303783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.304064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.304365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.306610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:33.314550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.316377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.328389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 323/643] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:33.329264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d4e4-1ff3-4520-a603-0c6df9e0a69d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:33.331116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:33.331993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:42:33.333003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:33.335056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 321/644] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:33.335959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.448358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d4e4-1ff3-4520-a603-0c6df9e0a69d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.452837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa7048-0263-48a6-8f40-25cef46dd6d3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.475840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:33.489276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.489894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.490178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.490511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.491109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:33.491412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.492811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.493660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.494267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.494549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.494824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.495468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:33.495773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.497156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.509216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 324/645] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:33.509216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.509419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 322/646] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:33.510163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.568431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:33.613545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3bf5-a953-408e-97b8-cc44bed1600e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.625659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc1851-82a6-4f68-b350-c18d0b3d7602 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.649170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868970b3-8354-4077-bf94-dde231456889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.649929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868970b3-8354-4077-bf94-dde231456889 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.650306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868970b3-8354-4077-bf94-dde231456889 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.650700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868970b3-8354-4077-bf94-dde231456889 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.662155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868970b3-8354-4077-bf94-dde231456889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:33.663147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14695-4880-4227-8492-706276860912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.663818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14695-4880-4227-8492-706276860912 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.664046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14695-4880-4227-8492-706276860912 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.664345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14695-4880-4227-8492-706276860912 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.665068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14695-4880-4227-8492-706276860912 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:33.665375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14695-4880-4227-8492-706276860912 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.666803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14695-4880-4227-8492-706276860912 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-868970b3-8354-4077-bf94-dde231456889 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:33.666909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:33.667756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 323/647] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:33.675004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868970b3-8354-4077-bf94-dde231456889 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.696671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 325/648] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:33.696671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14695-4880-4227-8492-706276860912 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.740473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:33.785144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868970b3-8354-4077-bf94-dde231456889 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.799614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:33.802793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14695-4880-4227-8492-706276860912 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.823497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.824138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.824443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.824715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.833334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.834789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.841083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:33.841896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:33.842250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:33.842603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:33.846124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:33.846980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:42:33.847882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:33.849731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:33.851171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:33.851784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 324/649] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:33.852636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:33.862849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:33.863721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 326/650] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:33.869703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:33.970213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aea2ee-0cab-4361-ab92-fb620f80bbe8 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.971428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25d079-fdc4-4152-9072-beedfb75f714 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:33.973948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.009104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.009713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.009993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.010094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.010269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.010698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.010921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:34.011017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.011224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.011326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.012722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.024047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 325/651] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:34.024806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.060112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.093395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.095321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.128363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77931f-7d80-430c-95df-a17374460d5b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.162496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.163098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.163380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.163653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.164410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:34.164718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.166152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.183853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 326/652] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:34.184699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.207161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 327/653] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:34.208011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.210246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.285664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.293659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdde308-b129-46a1-b6cc-e107751750c0 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.311136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d87cc-a8fb-4d4f-96f4-4d07e06b453d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.328251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.328898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.329175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.329445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.330135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:34.330439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.331859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.343459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 327/654] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:34.344407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.348625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.349160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.349438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.349707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.360480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.361930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:34.376703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:34.377614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:34.378913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:34.378913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:34.378913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:34.378913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 328/655] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:34.378913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.441461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.449159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c645c-da56-4962-8505-f6243029ebaa None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.482825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.483461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.483698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.483971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.488799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611a4d-d368-4d27-bef5-8e3588857790 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.509628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.522427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a40d6d-10a9-499b-b7be-75b4244d834e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.523051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a40d6d-10a9-499b-b7be-75b4244d834e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.523326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a40d6d-10a9-499b-b7be-75b4244d834e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.523594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a40d6d-10a9-499b-b7be-75b4244d834e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.534556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a40d6d-10a9-499b-b7be-75b4244d834e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.536060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a40d6d-10a9-499b-b7be-75b4244d834e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-19a40d6d-10a9-499b-b7be-75b4244d834e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:34.550798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:34.551868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:34.552778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:34.552778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:34.552778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:34.552778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 329/656] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:34.553417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b3be8-8781-4cf1-a845-ca36b900ea69 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.553417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b3be8-8781-4cf1-a845-ca36b900ea69 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.553605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b3be8-8781-4cf1-a845-ca36b900ea69 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.554829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 330/657] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:34.555529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b3be8-8781-4cf1-a845-ca36b900ea69 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.558837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.560469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.657093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 328/658] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:34.657945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.659379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b3be8-8781-4cf1-a845-ca36b900ea69 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.666712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.695766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.696407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.696681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.696948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.697662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=86456c6dbe4e454e9f255960b70c0832) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:34.697965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.699376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.705226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 331/659] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/role_assignments?user.id=86456c6dbe4e454e9f255960b70c0832&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:34.706081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.714116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.775843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c494-0596-41f1-acf7-c372ad106870 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.818630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25795d-b78d-406c-b66f-faf475c168a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.819609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25795d-b78d-406c-b66f-faf475c168a9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.820065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25795d-b78d-406c-b66f-faf475c168a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.820573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25795d-b78d-406c-b66f-faf475c168a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.821405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25795d-b78d-406c-b66f-faf475c168a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:34.821900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25795d-b78d-406c-b66f-faf475c168a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.823484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25795d-b78d-406c-b66f-faf475c168a9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.836567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 329/660] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:34.837758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83568bd0-c51a-4a43-b453-b0b6ef969d40 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:34.838307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18719dd-0380-4e9b-b29d-89a97d6df8ff None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.838768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18719dd-0380-4e9b-b29d-89a97d6df8ff None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.839202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18719dd-0380-4e9b-b29d-89a97d6df8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.861253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e18719dd-0380-4e9b-b29d-89a97d6df8ff None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:34.862447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e18719dd-0380-4e9b-b29d-89a97d6df8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:34.884528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:34.885213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:34.885506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:34.885756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:34.886396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:34.886696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:34.888066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:34.900981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 332/661] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:34.901812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.940787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 330/662] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:34.940787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18719dd-0380-4e9b-b29d-89a97d6df8ff None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:34.948263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:34.997651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.019162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889ceb3-b8aa-4570-a4bd-d1a728d7875b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.055681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.056353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.056634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.056910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.058115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18719dd-0380-4e9b-b29d-89a97d6df8ff None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.067630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.069195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:35.087672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:35.089684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:35.090575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:35.090575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:35.090575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:35.090575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 333/663] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:35.090575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.099960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.100662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.100945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.101217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.115181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.117300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:35.133265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:35.134489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 331/664] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:35.135567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.191476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.212108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bcf28-d6e2-4ba7-b446-83ffa5fe16fe None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.249091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787ff0a-912b-49fa-8d52-b42f10554b94 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.250113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.251274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.251554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.251833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.252850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.259047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.260523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:35.272664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:35.273554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:35.274415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 334/665] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:35.274892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.292926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.293554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.293866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.294103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.305799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:35.311594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:35.312766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:35.314077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:35.318503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 332/666] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:35.319389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.381312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119493c-e7aa-4f2b-bcaa-cbadc21ea365 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.420973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.421802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.422262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.422682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.423581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:35.424030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.424561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09b9df-7055-4a94-87b1-b8cc3a268e84 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.425760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.438998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 335/667] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:35.440112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.443051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.476773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.477837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.478118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.478390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.479129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:35.479430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.480893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.501599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 333/668] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:35.502433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.529573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.568440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34881a0c-e0ac-4a45-89e5-cc13f47739ff None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.605620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eaa0a-ff06-4a05-a390-cda0b55574e4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.610859 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.611472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.611746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.612009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.612746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:35.613256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.614449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.631581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 336/669] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:35.631581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.648792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.649406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.649688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.649961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.650621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:35.650926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.652327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.665067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 334/670] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:35.665067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.700974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.742735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21bc15-1aa6-4801-b44a-ec4ece3ca796 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.755958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.772551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb465a7-7883-40c1-af8a-6b70da4c1229 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.783351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.784278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.784559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.784831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.793173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.795030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.806392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.807017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.807296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.807572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:35.810198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:35.811072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:35.811926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:35.813464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 337/671] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:35.813803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.884998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.886595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.923730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58700cbd-c751-4821-a92e-714cca649cba None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:35.946729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:35.960829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:35.961509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:35.961791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:35.962081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:35.963433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=770e3b2a34d74195bcdf1d0b6f696cda) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:35.963433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:35.964584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:35.970690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 338/672] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/role_assignments?user.id=770e3b2a34d74195bcdf1d0b6f696cda&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:35.971281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:35.992733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:36.020420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 335/673] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:36.021734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.107931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd5165-fc0c-4988-bdaf-4f81ba4e3c13 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.152947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.153506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.153788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.154061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.154390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953837ab-786a-47f8-9a95-77fdc7260693 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.169238 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.171022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:36.186932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:36.187885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:36.189819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:36.189819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:36.189819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:36.189819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 339/674] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:36.189819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.208223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.208830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.209097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.209370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.210066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:36.210366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.211790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.231017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 336/675] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:36.231930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.236305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:36.287002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:36.309717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9427cfc-4406-4e6f-92f9-544e2c5b6f4b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.351553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed44851-7693-4b3f-907d-534cfcc64b97 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.352275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.353014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.353392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.354033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.368871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.371201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:36.391744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:36.392750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:36.393629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:36.393629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:36.393629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:36.393794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.393794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.393794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.393794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.394565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:36.394662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.396236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.396686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 340/676] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:36.397668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.422099 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 337/677] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:36.423117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.522656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:36.537384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0b30f-1eee-4614-8881-7204f4551ef4 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.541110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b8a2f-b8e4-43a9-aba3-1457d3df024a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.545242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:36.579534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.580292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.580817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.580817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.581731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=88c76a4a9747469d88a0a1a5748452a1) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:36.582051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.583472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.590572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.591434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.591915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.592450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.593310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:36.593811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.595588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 341/678] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/role_assignments?user.id=88c76a4a9747469d88a0a1a5748452a1&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:36.596363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.596549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.611279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 338/679] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:36.617589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.728128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e387f-d8de-41e1-8dc3-16a8ca3c0327 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.767003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10eb4f-5024-4079-9e89-a958d44989a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.768096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10eb4f-5024-4079-9e89-a958d44989a4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.768599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10eb4f-5024-4079-9e89-a958d44989a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.769068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10eb4f-5024-4079-9e89-a958d44989a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.777123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10eb4f-5024-4079-9e89-a958d44989a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.778982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10eb4f-5024-4079-9e89-a958d44989a4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.786939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbd621-e5d5-44e2-a410-cd98b2e9b345 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-9d10eb4f-5024-4079-9e89-a958d44989a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:36.792622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:36.793539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:36.794403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:36.797235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 339/680] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:36.802393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddc30-d196-49e7-8f80-22967d093da0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.802681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddc30-d196-49e7-8f80-22967d093da0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.802958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddc30-d196-49e7-8f80-22967d093da0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.804567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 340/681] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:36.805281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ddc30-d196-49e7-8f80-22967d093da0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.836434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcff23b-0c83-4887-aaf9-7e3517c87e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.837087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcff23b-0c83-4887-aaf9-7e3517c87e23 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.837365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcff23b-0c83-4887-aaf9-7e3517c87e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.837638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcff23b-0c83-4887-aaf9-7e3517c87e23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.844540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:36.846322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcff23b-0c83-4887-aaf9-7e3517c87e23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.847807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcff23b-0c83-4887-aaf9-7e3517c87e23 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-afcff23b-0c83-4887-aaf9-7e3517c87e23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:36.860617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:36.861729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:36.862650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 342/682] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:36.863737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81118d91-9d12-4654-9219-a660208dee32 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.863737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81118d91-9d12-4654-9219-a660208dee32 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.863901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81118d91-9d12-4654-9219-a660208dee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.887463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-81118d91-9d12-4654-9219-a660208dee32 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:36.889305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-81118d91-9d12-4654-9219-a660208dee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:36.902816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:36.936746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ddc30-d196-49e7-8f80-22967d093da0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:36.983893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 343/683] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:36.985174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81118d91-9d12-4654-9219-a660208dee32 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:36.987927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:36.988592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:36.989715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:36.989988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:36.990684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:36.990986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:36.993138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:37.016582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 341/684] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:37.016582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:37.114515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81118d91-9d12-4654-9219-a660208dee32 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.119098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:37.138804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b191f4-7d2b-4eab-a4ff-93078ecba6de None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.163383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:37.164282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:37.164567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:37.164837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:37.165987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:37.166297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:37.168173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:37.178450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:37.179179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:37.179566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:37.179842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:37.192239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 344/685] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 10 16:42:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:37.193356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:37.196826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:37.201686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:37.202805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:37.203671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 342/686] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:37.205049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:37.206352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:37.323029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39dd6e-27ec-4898-95ed-2580aa40bb23 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.329258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd81e1-bd54-4d88-ad4f-327756b2b015 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.362758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:37.363831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:37.364410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:37.364879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:37.376174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:37.376174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:37.376174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:37.376174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:37.376659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=256af8436dd842fdb4ebe90ec9be26ae) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:37.380166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:37.380166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:37.381626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:37.384626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 345/687] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:37 2022] GET /identity/v3/role_assignments?user.id=256af8436dd842fdb4ebe90ec9be26ae&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:37.385542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:37.447283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:37.448913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:37.488541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:37.547073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9244dd-4b27-4355-9725-aedd4bebd0fb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.586925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:37.587690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:37.588204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:37.588585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:37.589497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:37.589921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:37.591438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:37.606415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 346/688] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:37.608001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:37.637210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 343/689] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:37.644540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:37.723483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:37.774524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de11c-b836-4987-ab2e-9409e75e5c27 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.815488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:37.816420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:37.816698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:37.816970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:37.827999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:37.829549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:37.845966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:37.847124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:37.848235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:37.848235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:37.848235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:37.848235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 344/690] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:37.854931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:37.863846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15748e-de4c-4658-a6d2-b10268105036 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.949470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:37.964542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:37.968766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2351c-8c21-4cb4-addb-7261e1761239 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:37.993462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:37.994125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:37.994416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:37.994655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.007149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:38.008336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.008785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.009210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.010569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:38.011027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:38.012597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:38.027250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 345/691] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:38.032449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:38.032449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:38.033771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:38.061070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:38.062348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:38.063598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:38.063598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:38.063598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:38.063598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 347/692] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:38.068253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:38.149863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d14839-a188-4d81-bc6d-0d2658a06e29 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:38.195746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6a18a-6964-4669-96d9-51a3432446eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:38.198350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a18a-6964-4669-96d9-51a3432446eb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.198785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a18a-6964-4669-96d9-51a3432446eb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.199206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a18a-6964-4669-96d9-51a3432446eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.215510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6a18a-6964-4669-96d9-51a3432446eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:38.217371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6a18a-6964-4669-96d9-51a3432446eb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-6ec6a18a-6964-4669-96d9-51a3432446eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:38.238490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:38.239643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:38.240770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:38.249415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 346/693] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:38.256447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f11697-0c8b-4b6f-92f3-7d247f243010 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.256737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f11697-0c8b-4b6f-92f3-7d247f243010 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.257013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f11697-0c8b-4b6f-92f3-7d247f243010 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.259918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 347/694] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:38.261987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f11697-0c8b-4b6f-92f3-7d247f243010 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:38.273463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c6cc9-90df-4eec-b67e-fe33f8749c10 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:38.304500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:38.311869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc27111-f382-48ac-8095-b27d2b425bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:38.312546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc27111-f382-48ac-8095-b27d2b425bbf None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.313357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc27111-f382-48ac-8095-b27d2b425bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.313636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc27111-f382-48ac-8095-b27d2b425bbf None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.315169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc27111-f382-48ac-8095-b27d2b425bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:38.315169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc27111-f382-48ac-8095-b27d2b425bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:38.316403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc27111-f382-48ac-8095-b27d2b425bbf None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:38.329476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 348/695] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:38.330723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73a8a8-e5cc-4b31-bab6-5975b720111b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.331011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73a8a8-e5cc-4b31-bab6-5975b720111b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.331285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73a8a8-e5cc-4b31-bab6-5975b720111b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.363460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ee73a8a8-e5cc-4b31-bab6-5975b720111b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:38.364722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ee73a8a8-e5cc-4b31-bab6-5975b720111b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:38.404498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f11697-0c8b-4b6f-92f3-7d247f243010 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:38.452738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:38.469547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:38.470936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.471425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.472034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.473175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:38.473687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:38.475792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:38.497798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 348/696] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:38.499063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:38.534775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 349/697] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:38.534775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73a8a8-e5cc-4b31-bab6-5975b720111b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:38.612691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:38.655845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8727b-8744-4318-a272-ba7833b34a97 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:38.716929 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73a8a8-e5cc-4b31-bab6-5975b720111b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:38.738875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:38.739518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.739799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.740076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.748069 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:38.755360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:38.761849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:38.762759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:38.763383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:38.763860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:38.764908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=f64c104787604ea4b22e2af2507b3f0a) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:38.765476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:38.767191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:38.780567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 350/698] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:38 2022] GET /identity/v3/role_assignments?user.id=f64c104787604ea4b22e2af2507b3f0a&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:38.782243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:38.803795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:38.804838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:38.805750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:38.820201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 349/699] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:42:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:38.820201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:38.864165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:39.068984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700d1f0-52d1-4bbf-9b76-81f4a4e5fa4e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:39.091545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752e415-7524-4719-ae36-68e492c0edbe None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:39.110698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:39.111340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.111615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.111883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.113291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:39.114671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:39.127208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:39.128319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:39.129190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 351/700] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Aug 10 16:42:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:39.135043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:39.152690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf1076-41c5-4e16-b629-b13dab8408b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:39.153300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf1076-41c5-4e16-b629-b13dab8408b4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.153585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf1076-41c5-4e16-b629-b13dab8408b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.153904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf1076-41c5-4e16-b629-b13dab8408b4 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.154583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf1076-41c5-4e16-b629-b13dab8408b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:39.155515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf1076-41c5-4e16-b629-b13dab8408b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:39.157806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf1076-41c5-4e16-b629-b13dab8408b4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:39.161635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:39.186854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 350/701] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:42:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:39.189308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98df9df-d0b3-4c2b-ad6d-54a8227af8e7 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.189787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98df9df-d0b3-4c2b-ad6d-54a8227af8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.190274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98df9df-d0b3-4c2b-ad6d-54a8227af8e7 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.192385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 351/702] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:39.193351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98df9df-d0b3-4c2b-ad6d-54a8227af8e7 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:39.265185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:39.271833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1e59d-7055-41bc-b87a-1c4bf028176d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:39.318488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379e11a-8d72-413e-b743-430facec06d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:39.319196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379e11a-8d72-413e-b743-430facec06d6 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.319660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379e11a-8d72-413e-b743-430facec06d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.319947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379e11a-8d72-413e-b743-430facec06d6 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.320617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379e11a-8d72-413e-b743-430facec06d6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:39.321329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379e11a-8d72-413e-b743-430facec06d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:39.323083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379e11a-8d72-413e-b743-430facec06d6 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:39.329979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 352/703] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 10 16:42:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:39.331179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851cb18-b33b-4f15-87e5-fc8a33384e08 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.331465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851cb18-b33b-4f15-87e5-fc8a33384e08 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.331740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851cb18-b33b-4f15-87e5-fc8a33384e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.333277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98df9df-d0b3-4c2b-ad6d-54a8227af8e7 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:39.358991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8851cb18-b33b-4f15-87e5-fc8a33384e08 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:39.361837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8851cb18-b33b-4f15-87e5-fc8a33384e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:39.375827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698713ed-6df4-4388-a827-95786730900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:39.376498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698713ed-6df4-4388-a827-95786730900f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.376778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698713ed-6df4-4388-a827-95786730900f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.377135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698713ed-6df4-4388-a827-95786730900f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.445553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:39.455586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698713ed-6df4-4388-a827-95786730900f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:39.457186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698713ed-6df4-4388-a827-95786730900f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:39.460322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 353/704] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:39.461867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851cb18-b33b-4f15-87e5-fc8a33384e08 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:39.568468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 352/705] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:42:39.569514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698713ed-6df4-4388-a827-95786730900f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:39.578794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:39.590136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851cb18-b33b-4f15-87e5-fc8a33384e08 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:39.629267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:39.630038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.630413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.630791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.631535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:39.631933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:39.633486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:39.641075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 354/706] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 10 16:42:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:39.642159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:39.712181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:39.728079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698713ed-6df4-4388-a827-95786730900f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:39.824247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7d889-0974-47ca-be31-00ec50046281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:39.824811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d889-0974-47ca-be31-00ec50046281 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.825088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d889-0974-47ca-be31-00ec50046281 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.825360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d889-0974-47ca-be31-00ec50046281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.831369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc72e4b-e0a7-4946-89e9-eb0670bd67a9 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:39.851181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7d889-0974-47ca-be31-00ec50046281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:39.853187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7d889-0974-47ca-be31-00ec50046281 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-9cf7d889-0974-47ca-be31-00ec50046281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:39.871555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:39.872665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:39.873518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:39.873518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:39.873518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:39.873518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 353/707] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:39.874325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247df0fc-aae2-4ca7-9fc5-582d01688868 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.874602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247df0fc-aae2-4ca7-9fc5-582d01688868 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.874870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247df0fc-aae2-4ca7-9fc5-582d01688868 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.876931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 354/708] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:39.880200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247df0fc-aae2-4ca7-9fc5-582d01688868 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:39.889836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d2115-d2a7-49e7-a584-e390c38ce2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:39.890480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d2115-d2a7-49e7-a584-e390c38ce2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.890766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d2115-d2a7-49e7-a584-e390c38ce2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.891597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d2115-d2a7-49e7-a584-e390c38ce2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.912172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d2115-d2a7-49e7-a584-e390c38ce2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:39.912172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d2115-d2a7-49e7-a584-e390c38ce2f7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-f67d2115-d2a7-49e7-a584-e390c38ce2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:39.922382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:39.923357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:39.924255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:39.924255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:39.924255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:39.924255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 355/709] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:39.928170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89c8e3-0e41-4b87-a21d-5f087cdb5aa0 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:39.928170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89c8e3-0e41-4b87-a21d-5f087cdb5aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:39.928170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89c8e3-0e41-4b87-a21d-5f087cdb5aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:39.947860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8d89c8e3-0e41-4b87-a21d-5f087cdb5aa0 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:39.949175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8d89c8e3-0e41-4b87-a21d-5f087cdb5aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:39.951403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:40.024981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 356/710] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:40.025907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247df0fc-aae2-4ca7-9fc5-582d01688868 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.026102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89c8e3-0e41-4b87-a21d-5f087cdb5aa0 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.072344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.073357 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.073647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.073924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.074900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:40.075150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.076862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.094024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 355/711] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:40.094857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.107115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:40.152864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89c8e3-0e41-4b87-a21d-5f087cdb5aa0 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.191400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.192209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.192572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.192969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.193710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:40.194068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.195503 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.205504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:40.215982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 357/712] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:40.215982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.221673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ba69f-907d-4f03-8ab4-e9a43931697d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.262301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.262935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.263209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.263475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] PATH_INFO: `/v3/services/network` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.264047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.265433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:40.282560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:40.283798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:40.285024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:40.292416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 356/713] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:40 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:40.292416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.371836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e99dd-caa2-47fe-afe5-b4e514d6c977 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.372626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:40.409101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa7133-f481-40c1-886c-d4d9cacc3728 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.424006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.425437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.426133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.426581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.427656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.429720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.447742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.448580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.448877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.449146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.457957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.459593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.461392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 358/714] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:42:40 2022] POST /identity/v3/users => generated 254 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:40.462543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:40.483581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:40.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:40.485565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:40.491265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 357/715] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:40.496928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.534946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:40.592874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a3a7-eed9-45c6-8a06-15740f71b544 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.633724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.634709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.635201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.635678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.636580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] RBAC: Adding query filter params (name=network) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:40.637086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.639050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.650616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 359/716] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:42:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:40.652287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.736556 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be98b-3b06-4afa-aafa-236be657d39e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.763532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8605b-6169-4406-8bcf-d7964f18b93d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.769564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:40.776870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.778240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.778522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.778796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.779603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:40.779911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.781674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.788579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:40.797005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 358/717] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:40.802391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:40.809268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559034f9-54a8-4886-be27-8b21a3a2df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.810021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559034f9-54a8-4886-be27-8b21a3a2df29 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.810401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559034f9-54a8-4886-be27-8b21a3a2df29 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.811463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559034f9-54a8-4886-be27-8b21a3a2df29 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.812231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559034f9-54a8-4886-be27-8b21a3a2df29 None admin] RBAC: Adding query filter params (type=network) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:40.812632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559034f9-54a8-4886-be27-8b21a3a2df29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.814731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559034f9-54a8-4886-be27-8b21a3a2df29 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.824278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 360/718] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:42:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:40.921702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65da2af-3eec-476a-a7bf-3d512fba2c96 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:40.961827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1807d8-d137-465a-a8d8-d073b27e4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:40.962701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1807d8-d137-465a-a8d8-d073b27e4258 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:40.963124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1807d8-d137-465a-a8d8-d073b27e4258 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:40.963535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1807d8-d137-465a-a8d8-d073b27e4258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:40.964472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1807d8-d137-465a-a8d8-d073b27e4258 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=86456c6dbe4e454e9f255960b70c0832) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:40.964926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1807d8-d137-465a-a8d8-d073b27e4258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:40.966546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1807d8-d137-465a-a8d8-d073b27e4258 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:40.974258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 359/719] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:40 2022] GET /identity/v3/role_assignments?user.id=86456c6dbe4e454e9f255960b70c0832&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:41.299872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7027da9-0482-4b44-9218-58df4314406c None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:41.300434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7027da9-0482-4b44-9218-58df4314406c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:41.300434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7027da9-0482-4b44-9218-58df4314406c None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:41.302051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 361/720] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:41.307519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3560598-ac3b-46cb-b6d9-14ff006080ab None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:41.308035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3560598-ac3b-46cb-b6d9-14ff006080ab None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:41.308554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3560598-ac3b-46cb-b6d9-14ff006080ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:41.331296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e3560598-ac3b-46cb-b6d9-14ff006080ab None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:41.332535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e3560598-ac3b-46cb-b6d9-14ff006080ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:41.399824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:41.422180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 360/721] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:41.458551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7027da9-0482-4b44-9218-58df4314406c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:41.594564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7027da9-0482-4b44-9218-58df4314406c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:41.640789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:41.642014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:41.642291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:41.642555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:41.653692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:41.659176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:41.660092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:41.661163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:41.665161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 362/722] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:41.666363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3560598-ac3b-46cb-b6d9-14ff006080ab None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:41.807803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3560598-ac3b-46cb-b6d9-14ff006080ab None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:41.849735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce23787-9365-4b35-9952-d61ed89a0fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:41.850275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23787-9365-4b35-9952-d61ed89a0fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:41.850549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23787-9365-4b35-9952-d61ed89a0fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:41.850818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23787-9365-4b35-9952-d61ed89a0fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:41.851477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23787-9365-4b35-9952-d61ed89a0fb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:41.851777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23787-9365-4b35-9952-d61ed89a0fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:41.853170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23787-9365-4b35-9952-d61ed89a0fb5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:41.867179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 361/723] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:41.872693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:41.895133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ff61-5254-49a9-977f-128f89eddf7a None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:41.895421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ff61-5254-49a9-977f-128f89eddf7a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:41.895693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ff61-5254-49a9-977f-128f89eddf7a None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:41.897306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 362/724] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:41.902124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbea7-a739-49e1-9f71-d43b6abc7bed None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:41.902406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbea7-a739-49e1-9f71-d43b6abc7bed None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:41.902678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbea7-a739-49e1-9f71-d43b6abc7bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:41.921159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-337dbea7-a739-49e1-9f71-d43b6abc7bed None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:41.922436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-337dbea7-a739-49e1-9f71-d43b6abc7bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:41.932530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:41.982556 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:42.018922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9570cf-c24d-4fab-a425-ea7946b9cce1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:42.025836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 363/725] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:42:42.051546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbea7-a739-49e1-9f71-d43b6abc7bed None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:42.063808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:42.067404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:42.067404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:42.067404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:42.077370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:42.079796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:42.096651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:42.097616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:42.098506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:42.098506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:42.098506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:42.098506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 363/726] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:42.103142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:42.216466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:42.223363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbb5ab-511d-4805-b7ae-3eba240ca144 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:42.245970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbea7-a739-49e1-9f71-d43b6abc7bed None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:42.265097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:42.265708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:42.266555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:42.266969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:42.267624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:42.267935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:42.269646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:42.284659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 364/727] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:42.286797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:42.287679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:42.288206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:42.288683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:42.289561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:42.290028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:42.291460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:42.310334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 364/728] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:42:42 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:42.407149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe7a73-bf1e-443e-b638-3786d506ac89 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:42.456429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:42.457074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:42.457405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:42.457686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:42.465843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:42.467307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:42.480102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:42.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:42.482645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 365/729] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:42.487429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:42.535144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:42.614530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237e7f9-c9e1-4b6f-9378-eed922b37a96 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:42.652201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:42.653165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:42.653676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:42.654165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:42.655100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:42.655629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:42.657959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:42.681347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 365/730] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 10 16:42:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:42.686618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:42.707095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:42.819086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b21da-0f18-44fc-a951-1a9b30a50ff0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:42.854596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e0bdc-cfe7-4ed3-9952-4b53f1aeadf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:42.855219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e0bdc-cfe7-4ed3-9952-4b53f1aeadf0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:42.855500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e0bdc-cfe7-4ed3-9952-4b53f1aeadf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:42.855774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e0bdc-cfe7-4ed3-9952-4b53f1aeadf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:42.956358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e0bdc-cfe7-4ed3-9952-4b53f1aeadf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:42.957829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e0bdc-cfe7-4ed3-9952-4b53f1aeadf0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:42.974969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:43.095925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 366/731] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:42 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:42:43.101115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:43.195218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b2438-c4df-42d5-b096-56bfa34fce36 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.195510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b2438-c4df-42d5-b096-56bfa34fce36 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.195791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b2438-c4df-42d5-b096-56bfa34fce36 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.197539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 367/732] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:43.209455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ffde2-9b43-4373-b87c-91f082e77fc9 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.209706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ffde2-9b43-4373-b87c-91f082e77fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.209945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ffde2-9b43-4373-b87c-91f082e77fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.243762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d98ffde2-9b43-4373-b87c-91f082e77fc9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:43.245542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d98ffde2-9b43-4373-b87c-91f082e77fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:43.247107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:43.255468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e146c7-8bf0-49a2-b7d8-ebe323a98de7 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:43.299596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:43.300267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.300713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.300990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.315189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:43.317013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:43.319385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 368/733] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 16:42:43.320937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf7ae-313a-41e2-95d1-a34f98c5b121 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.321522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf7ae-313a-41e2-95d1-a34f98c5b121 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.321990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf7ae-313a-41e2-95d1-a34f98c5b121 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.323623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 369/734] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:43.332875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2833-b3a7-41f1-9d8a-e92ad4205ebc None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.333423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2833-b3a7-41f1-9d8a-e92ad4205ebc None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.333925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2833-b3a7-41f1-9d8a-e92ad4205ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:43.335965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:43.336874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:43.337731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:43.337731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:43.337731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:43.337731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 366/735] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:43.342632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:43.356678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4fbb2833-b3a7-41f1-9d8a-e92ad4205ebc None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:43.358321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4fbb2833-b3a7-41f1-9d8a-e92ad4205ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:43.395324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:43.440357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 370/736] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:42:43.441864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb2833-b3a7-41f1-9d8a-e92ad4205ebc None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:43.489732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf624da5-d317-468d-b39f-4ab2dd10c8e4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:43.537698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:43.538591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.538874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.539154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.540054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:43.540388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:43.542217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:43.559662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb2833-b3a7-41f1-9d8a-e92ad4205ebc None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:43.568326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 367/737] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:43.569167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:43.579916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:43.608730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:43.611181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.611693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.612183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.626333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:43.632845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:43.633818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:43.634620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:43.637215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 371/738] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:43.638139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:43.680276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:43.689133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60faabf-9eac-45ad-b777-4c728a678e90 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:43.726695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:43.727474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.727848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.728243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.729085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:43.730635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:43.760269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 368/739] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:42:43 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:43.760269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:43.804094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157ad12-39a6-4558-bc64-4f422f561915 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:43.887744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020789ac-a7a1-42b1-b2c2-4515b03cf452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:43.888421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020789ac-a7a1-42b1-b2c2-4515b03cf452 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.888698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020789ac-a7a1-42b1-b2c2-4515b03cf452 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.888971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020789ac-a7a1-42b1-b2c2-4515b03cf452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.893327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:43.903813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020789ac-a7a1-42b1-b2c2-4515b03cf452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:43.911157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020789ac-a7a1-42b1-b2c2-4515b03cf452 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-020789ac-a7a1-42b1-b2c2-4515b03cf452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:43.927497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:43.928667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:43.929661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 372/740] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:43.930604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7bbc9-fe5f-48d3-bc57-97377dc9c245 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:43.930891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7bbc9-fe5f-48d3-bc57-97377dc9c245 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:43.931164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7bbc9-fe5f-48d3-bc57-97377dc9c245 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:43.932729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 373/741] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:43.935727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7bbc9-fe5f-48d3-bc57-97377dc9c245 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:43.950306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25151c-9291-4a9f-8f5d-d89059b1fcc7 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:43.999484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce9450-94e7-41e3-b8c8-831621e181e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.000105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce9450-94e7-41e3-b8c8-831621e181e1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.000440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce9450-94e7-41e3-b8c8-831621e181e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.000709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce9450-94e7-41e3-b8c8-831621e181e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.001374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce9450-94e7-41e3-b8c8-831621e181e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:44.001672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce9450-94e7-41e3-b8c8-831621e181e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:44.003043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce9450-94e7-41e3-b8c8-831621e181e1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:44.013068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 369/742] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:44.014156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815ec7-4152-497c-88de-50242d868c59 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.014443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815ec7-4152-497c-88de-50242d868c59 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.014717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815ec7-4152-497c-88de-50242d868c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.034747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ec815ec7-4152-497c-88de-50242d868c59 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:44.035959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ec815ec7-4152-497c-88de-50242d868c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:44.063383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:44.083297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7bbc9-fe5f-48d3-bc57-97377dc9c245 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.120883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 370/743] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:44.121579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.122175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.122454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.122723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.123362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec815ec7-4152-497c-88de-50242d868c59 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:44.123468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:44.123708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:44.125309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:44.137554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 374/744] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:44.144552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:44.195201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:44.243091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec815ec7-4152-497c-88de-50242d868c59 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.276123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d7dc9-391e-4050-9719-f37970837413 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.279266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.279948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.280267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.280542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.291505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:44.293085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:44.309416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:44.310413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:44.311402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:44.311402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:44.311402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:44.311402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 371/745] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:44.312050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:44.321172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.321806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.322079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.322355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.323199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=f64c104787604ea4b22e2af2507b3f0a) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:44.323502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:44.325088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:44.333713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 375/746] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/role_assignments?user.id=f64c104787604ea4b22e2af2507b3f0a&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:44.333713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:44.348603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:44.447835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a93c1-f101-4752-a6ac-bc02df4a6d59 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.462571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:44.501101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.501758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.502039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.502308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.522684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:44.547803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaca8e5-907a-47f8-9174-156f442568dc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.613460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.614163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.614465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.614715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.615397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:44.615701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:44.617213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:44.639814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 376/747] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:44.646451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:44.649839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:44.650760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:44.651645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 372/748] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:44.657161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:44.728744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:44.766132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210d29b-7a1a-4e64-8572-c8db7aeb6b2c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.811617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.812192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.812478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.813314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.825638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:44.827097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:44.852270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:44.853258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:44.854166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 377/749] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:44.861307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:44.882332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53f4b8-84f9-483c-961b-ece5bed042a7 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.958636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:44.959873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:44.960393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:44.961019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:44.961890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:44.964498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:44.966744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:44.971909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b75ef-4c74-48b9-9a66-acc61c90bb8a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:44.978117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 373/750] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:44.983744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.021869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.036109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.050738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe58595-aa3f-4e23-980d-cf085a871cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.051366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58595-aa3f-4e23-980d-cf085a871cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.051649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58595-aa3f-4e23-980d-cf085a871cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.051922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58595-aa3f-4e23-980d-cf085a871cd7 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.053172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe58595-aa3f-4e23-980d-cf085a871cd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:45.054491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe58595-aa3f-4e23-980d-cf085a871cd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.056705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe58595-aa3f-4e23-980d-cf085a871cd7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.084296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 378/751] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:45.087344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406e04-0b60-4914-ab5f-b105fd012dd0 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.087641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406e04-0b60-4914-ab5f-b105fd012dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.087917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406e04-0b60-4914-ab5f-b105fd012dd0 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.092213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 379/752] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:45.093324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406e04-0b60-4914-ab5f-b105fd012dd0 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.122952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37326cff-949b-4ffa-a340-2b8aceb9c490 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.159248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9a958-30f5-43b4-b39f-dce25856769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.159616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9a958-30f5-43b4-b39f-dce25856769e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.159897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9a958-30f5-43b4-b39f-dce25856769e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.160204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9a958-30f5-43b4-b39f-dce25856769e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.171019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9a958-30f5-43b4-b39f-dce25856769e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.173227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9a958-30f5-43b4-b39f-dce25856769e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-fcd9a958-30f5-43b4-b39f-dce25856769e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:45.188340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:45.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:45.190228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:45.190228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:45.190228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:45.194380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 374/753] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:45.195651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157767d9-6bad-40fb-9247-9a945c92adb4 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.195941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157767d9-6bad-40fb-9247-9a945c92adb4 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.196241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157767d9-6bad-40fb-9247-9a945c92adb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.209698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406e04-0b60-4914-ab5f-b105fd012dd0 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.215831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-157767d9-6bad-40fb-9247-9a945c92adb4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:45.217124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-157767d9-6bad-40fb-9247-9a945c92adb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:45.249897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.250971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.251501 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.252080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.296269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 375/754] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:45.297888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157767d9-6bad-40fb-9247-9a945c92adb4 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.313869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.320063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.330507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.332497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.419514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157767d9-6bad-40fb-9247-9a945c92adb4 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.447572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 380/755] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:42:45.448602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.455816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.456465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.456748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.457023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.457667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:45.457971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.459352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.476015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 376/756] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:45.476915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.548318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.559840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.566171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59347ba3-f50d-42b0-8a57-49b4910fd5b7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.588920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96815ab-ae73-4df3-84ee-88b609b3bc1b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.607073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.608059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.608510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.608945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.609705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.611338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:45.621300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:45.622490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:45.623516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:45.626284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 381/757] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:45.627222 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.633267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.633994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.634343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.634685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.647516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.649095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:45.666563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:45.667748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:45.668911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:45.668911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:45.668911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:45.669732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 377/758] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:45.670710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.750159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32367b-04c1-464f-9955-361e1146d2ee None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.788708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c979ac-5388-4615-8ac4-1579da8f2355 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.794122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.794855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.795211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.795560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.805667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.807845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.820681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:45.822195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:45.823098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:45.824024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 382/759] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:45.829737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.838129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.838999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.839456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.839913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.841111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:45.841644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.843580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:45.849946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:45.856293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 378/760] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:45.857252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:45.944388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8922a5e-ada6-41ac-918c-4073c0eb0a1f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.968562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef014740-990f-4944-8078-22055e3ce5f1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:45.983864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:45.985280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:45.985708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:45.986146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:45.987029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:45.987588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:45.989205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.003909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 383/761] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.005205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.010381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.010530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.010903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.011254 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.012060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:46.012493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.014065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.040004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 379/762] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:42:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.041267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.086968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.099312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.124139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c970f-2b34-4f8e-b7ec-6a76d15d2ac5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.156444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d52b-abba-44f7-ade8-afc68886153f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.161873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.162488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.162768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.163041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.163669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:46.163976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.165375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.175221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 384/763] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.176075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.196791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d40fe-93fa-411a-adad-6d8f174d4fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.197348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d40fe-93fa-411a-adad-6d8f174d4fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.197623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d40fe-93fa-411a-adad-6d8f174d4fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.197888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d40fe-93fa-411a-adad-6d8f174d4fd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.205429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d40fe-93fa-411a-adad-6d8f174d4fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.207170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d40fe-93fa-411a-adad-6d8f174d4fd3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d92d40fe-93fa-411a-adad-6d8f174d4fd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:46.220530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:46.221422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:46.222331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:46.223341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 380/764] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:46.224872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c1ddd-d18c-4380-8baf-7beaf0c2399c None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.225163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c1ddd-d18c-4380-8baf-7beaf0c2399c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.225439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c1ddd-d18c-4380-8baf-7beaf0c2399c None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.226986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 381/765] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:46.227687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c1ddd-d18c-4380-8baf-7beaf0c2399c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.287790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb6fa4-dbc4-4a23-b4c0-f652af343671 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.322368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.323004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.323278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.323546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.326635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.336473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.337992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.350358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c1ddd-d18c-4380-8baf-7beaf0c2399c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:46.352905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:46.353866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:46.355004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:46.355004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:46.355004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:46.356462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 385/766] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:46.357378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.363770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.392777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d54a16-ecb2-4ed0-9f9c-ff4028921dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.393585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d54a16-ecb2-4ed0-9f9c-ff4028921dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.393694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d54a16-ecb2-4ed0-9f9c-ff4028921dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.394012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d54a16-ecb2-4ed0-9f9c-ff4028921dd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.394624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d54a16-ecb2-4ed0-9f9c-ff4028921dd6 None admin] RBAC: Adding query filter params (interface=public, service_id=b3430b397bc8401891574767ef242655, region_id=RegionOne) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:46.394933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d54a16-ecb2-4ed0-9f9c-ff4028921dd6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.396349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d54a16-ecb2-4ed0-9f9c-ff4028921dd6 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.405145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 382/767] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/endpoints?service_id=b3430b397bc8401891574767ef242655&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.406195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387c71e-6c76-48f5-90b3-7c8c8138dca7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.407104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387c71e-6c76-48f5-90b3-7c8c8138dca7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.407384 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387c71e-6c76-48f5-90b3-7c8c8138dca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.428669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7387c71e-6c76-48f5-90b3-7c8c8138dca7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:46.430386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7387c71e-6c76-48f5-90b3-7c8c8138dca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:46.478258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f95089-2e53-4eb8-a15a-e71a3b4378b0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.503855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 383/768] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:46.513338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387c71e-6c76-48f5-90b3-7c8c8138dca7 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.514029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.514910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.515403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.516441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.517336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:46.517844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.519548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.533005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 386/769] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.534185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.583906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.621763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387c71e-6c76-48f5-90b3-7c8c8138dca7 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.623268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.636230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8180fb9-5718-4321-a431-beca9e975c5c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.660680 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.661280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.661556 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.661824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.662451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:46.662738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.664096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.670155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.670836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.671113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.671384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.671978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.673340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.675014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 384/770] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.675791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.689178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 387/771] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/services => generated 772 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.689178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.793136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e40c-01be-488b-94e7-5a499e18e7a5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.799892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122099-73a0-4d4d-af97-9ba44358ea2f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:46.810327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.837140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d2cfc-2f39-4b79-b76b-b472e3a1028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.837759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2cfc-2f39-4b79-b76b-b472e3a1028d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.838084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2cfc-2f39-4b79-b76b-b472e3a1028d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.838315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2cfc-2f39-4b79-b76b-b472e3a1028d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.839375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d2cfc-2f39-4b79-b76b-b472e3a1028d None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=81fba04097094820be201ad24ed2658d) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:46.839375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d2cfc-2f39-4b79-b76b-b472e3a1028d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.840797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d2cfc-2f39-4b79-b76b-b472e3a1028d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.841803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd84ee-94ac-48cd-9092-f7a3cc2a24a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:46.842598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd84ee-94ac-48cd-9092-f7a3cc2a24a7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.843424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd84ee-94ac-48cd-9092-f7a3cc2a24a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.843424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd84ee-94ac-48cd-9092-f7a3cc2a24a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.843731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd84ee-94ac-48cd-9092-f7a3cc2a24a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:46.845608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd84ee-94ac-48cd-9092-f7a3cc2a24a7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:46.848297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 388/772] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/role_assignments?user.id=81fba04097094820be201ad24ed2658d&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:46.850138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e746b-6dfa-4e0f-b46f-b5fcbc9f14a4 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.850624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e746b-6dfa-4e0f-b46f-b5fcbc9f14a4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.851075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e746b-6dfa-4e0f-b46f-b5fcbc9f14a4 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.852681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 389/773] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:46.853617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e746b-6dfa-4e0f-b46f-b5fcbc9f14a4 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d8bd84ee-94ac-48cd-9092-f7a3cc2a24a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:46.855946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:46.856945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:46.857756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 385/774] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:46.858866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca60d-2260-4c62-9b04-b5f869261baf None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:46.859159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca60d-2260-4c62-9b04-b5f869261baf None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:46.859438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca60d-2260-4c62-9b04-b5f869261baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:46.881977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b38ca60d-2260-4c62-9b04-b5f869261baf None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:46.883227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b38ca60d-2260-4c62-9b04-b5f869261baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:46.906925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:46.962226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 386/775] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:46.963219 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca60d-2260-4c62-9b04-b5f869261baf None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:46.996236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e746b-6dfa-4e0f-b46f-b5fcbc9f14a4 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.036333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.037296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.037811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.038228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.046207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.048497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:47.061321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:47.062416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:47.063959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:47.066401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 390/776] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:47.071689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.072394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca60d-2260-4c62-9b04-b5f869261baf None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.108055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.108614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.108898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.109179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.110337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] RBAC: Adding query filter params (name=network) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:47.110642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.112014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.113797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:47.117109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 387/777] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:42:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:47.118276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.142245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:47.191271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89cbf9-5a33-4ad0-ad5e-8251995f76e3 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.228628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e2b60-5fa1-4822-bfda-222f8d484fff None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.229389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.230321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.230766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.231230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.244026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:47.251401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:47.252533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:47.253577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:47.256076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 391/778] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:47.257033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.267017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.267641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.267919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.268216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.268913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:47.269217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.270787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.288723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 388/779] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:47.288723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.389128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:47.396544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:47.398569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c02c7-f9a9-457c-8ee4-2e317748d8cd None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.424327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa117c8-77b6-489f-abf5-e303ecc0e854 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.447095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.448128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.448517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.448869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.449738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:47.450129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.451880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.458429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 392/780] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:47.459596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.485959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.486670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.487022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.487365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.488108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:47.488506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.489945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.501767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 389/781] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:47.502848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.601292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831333-fa02-478e-b116-be28a1f9b9f0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.616554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13b8a-ae5f-47e0-b393-55ba10248039 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.648498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.649543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.650063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.650574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/86456c6dbe4e454e9f255960b70c0832/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.652501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.653100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=86456c6dbe4e454e9f255960b70c0832, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.653299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.653677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.654051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.654853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] RBAC: Adding query filter params (interface=public, service_id=13283f1cdc16408bbf4817ecab55e3e8, region_id=RegionOne) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:47.655264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.655397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.657405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.667392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 390/782] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/endpoints?service_id=13283f1cdc16408bbf4817ecab55e3e8&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:47.667392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.677877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:47.680607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 393/783] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:47 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/86456c6dbe4e454e9f255960b70c0832/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:47.681861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.715344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:47.807887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f660427-e46d-42b7-a953-8058ce0cec92 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.811913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5a9d6-ffdc-479c-9ff7-db9fa18232dc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:47.853512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.854552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.854975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.855417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.868491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.870225 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:47.888791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:47.889990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:47.891533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:47.891533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:47.891533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:47.893710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 391/784] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:47.897238 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:47.897878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:47.898163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:47.898436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:47.899103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:47.899599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:47.900448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:47.919104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 394/785] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/services => generated 772 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:47.976278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:48.024983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e88589-203d-4d87-b7e7-a522c9af6399 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:48.062524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:48.063698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:48.063971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:48.064267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:48.064939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:48.065244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:48.066920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:48.081432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 392/786] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:48.086732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:48.202482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:48.220555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aad46-4def-4261-97ca-a78bae27517c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:48.254854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:48.255488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:48.255766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:48.256037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:48.265048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:48.267043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:48.278065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:48.278985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:48.279910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 395/787] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:48.284971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:48.447370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473b85a-1bdc-4be9-ab89-35152709e84a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:48.471398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:48.490500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:48.491380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:48.491735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:48.492080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:48.492962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:48.493345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:48.494870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:48.514133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 393/788] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 10 16:42:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:48.519718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:48.637134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925184a-18e3-4f18-b9e5-ce040743b7b6 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:48.680788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:48.681793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:48.682271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:48.682954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:48.696603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:48.698415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:48.701762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:48.716883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:48.717960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:48.718830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:48.718830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:48.718830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:48.721839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 396/789] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:48.727580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:48.850709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8848-5fba-4f30-a192-fd86b31e7ff1 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:48.887517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:48.888625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:48.889130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:48.889610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:48.890452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:48.891620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:48.893431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:48.910274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 394/790] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:48.918224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.080329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c7921-298a-4637-82fc-00a4ec254653 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:49.131232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46d143-ad33-438a-a6ec-1194252c5f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:49.131874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46d143-ad33-438a-a6ec-1194252c5f3e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.132171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46d143-ad33-438a-a6ec-1194252c5f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.132443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46d143-ad33-438a-a6ec-1194252c5f3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.142025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46d143-ad33-438a-a6ec-1194252c5f3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:49.143475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46d143-ad33-438a-a6ec-1194252c5f3e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-bb46d143-ad33-438a-a6ec-1194252c5f3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:49.158155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:49.159863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:49.161675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 397/791] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:49.170628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.178855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17db3a4-e82f-4c1d-a165-c5151f94941c None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.179278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17db3a4-e82f-4c1d-a165-c5151f94941c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.179624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17db3a4-e82f-4c1d-a165-c5151f94941c None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.181436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 398/792] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:49.189926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683fa3c-1cfe-4877-85ff-ce7430b00db4 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.189926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683fa3c-1cfe-4877-85ff-ce7430b00db4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.189926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683fa3c-1cfe-4877-85ff-ce7430b00db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.222722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d683fa3c-1cfe-4877-85ff-ce7430b00db4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:49.224060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d683fa3c-1cfe-4877-85ff-ce7430b00db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:49.256135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:49.288228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:49.318865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 399/793] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:42:49.320463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41404b-0319-4d90-853c-d9fc513189f0 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.320892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41404b-0319-4d90-853c-d9fc513189f0 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.321309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41404b-0319-4d90-853c-d9fc513189f0 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.322982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 400/794] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:49.338894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a686e5-17d4-45b9-b916-7d3052d09e86 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.339400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a686e5-17d4-45b9-b916-7d3052d09e86 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.339818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a686e5-17d4-45b9-b916-7d3052d09e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.361405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a3a686e5-17d4-45b9-b916-7d3052d09e86 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:49.363100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a3a686e5-17d4-45b9-b916-7d3052d09e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:49.374677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426e27d-4057-41ed-b97d-8cf2df86be9f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:49.435421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:49.436289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.436639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.437334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.438672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:49.439142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:49.441161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:49.462492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 395/795] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:49.462492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.464375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 401/796] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:49.465450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a686e5-17d4-45b9-b916-7d3052d09e86 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.577264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:49.585690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a46b-7a59-49e9-9ca7-5d4144808462 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:49.627754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a686e5-17d4-45b9-b916-7d3052d09e86 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:49.630402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:49.631262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.631734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.632233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.633023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:49.634660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:49.644630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:49.645594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:42:49.646399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:49.647109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:49.649481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 396/797] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Aug 10 16:42:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:49.650475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.681821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:49.682499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.682778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.683261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/f64c104787604ea4b22e2af2507b3f0a/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.686101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=f64c104787604ea4b22e2af2507b3f0a, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:49.688444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:49.712172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 402/798] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:49 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/f64c104787604ea4b22e2af2507b3f0a/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:49.712172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.823394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bf38-d884-4b23-897e-28f6aa3c3f1e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:49.841975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714623d-7c7d-4a2d-96f2-6c910164ea76 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:49.887108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:49.887769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.888056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.888397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.889052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:49.889357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:49.890835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:49.897766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 403/799] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 10 16:42:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:49.903425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.929123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:49.929771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:49.930051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:49.930327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:49.950181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:49.956192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:49.957149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:49.958039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:49.958681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 397/800] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:49.958751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:49.968366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:49.994696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:50.049652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf28c54-3456-4001-8b8c-deff1a21ccb1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:50.080604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808a429-2314-473b-9945-64b4828baf55 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:50.096014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:50.096672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.096991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.097271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.097905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:50.098264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:50.099867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:50.106073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 404/801] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 10 16:42:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:50.112891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:50.184883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:50.185515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.185792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.186233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.187061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:50.187511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:50.189644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:50.207809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 398/802] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:50.213611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:50.236026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4398d-d3b3-40b2-a33d-5ceaab6356d8 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:50.239570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:50.282370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5f3bd-f225-4f59-97e1-5a27d91b3567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:50.283509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f3bd-f225-4f59-97e1-5a27d91b3567 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.283931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f3bd-f225-4f59-97e1-5a27d91b3567 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.284640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f3bd-f225-4f59-97e1-5a27d91b3567 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.285909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5f3bd-f225-4f59-97e1-5a27d91b3567 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:50.287596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5f3bd-f225-4f59-97e1-5a27d91b3567 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:50.325136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 405/803] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:42:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:50.372486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824124e-7541-467d-860a-7f9e740558d8 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.372486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824124e-7541-467d-860a-7f9e740558d8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.372486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824124e-7541-467d-860a-7f9e740558d8 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.375413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 406/804] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:50.379893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e8e1d-ff94-42ac-86a5-df54238873a6 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.380205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e8e1d-ff94-42ac-86a5-df54238873a6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.380981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e8e1d-ff94-42ac-86a5-df54238873a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.388550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:50.421375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-bd5e8e1d-ff94-42ac-86a5-df54238873a6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:50.422965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042067b-a29f-469b-af75-a37195566ca3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:50.423706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bd5e8e1d-ff94-42ac-86a5-df54238873a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:50.470939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:50.471495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.471774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.472042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.485483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:50.487063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:50.502660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:50.503654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:50.504989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:50.504989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:50.504989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:50.504989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 399/805] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:50.509908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:50.539089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 407/806] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:50.540964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed462-b733-4983-bdcc-3ec223de4efa None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.541498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed462-b733-4983-bdcc-3ec223de4efa None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.542002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed462-b733-4983-bdcc-3ec223de4efa None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.544963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 408/807] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:50.552170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70fc1b-d688-4468-a52c-c3e3d9b437bd None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.552170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70fc1b-d688-4468-a52c-c3e3d9b437bd None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.552170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70fc1b-d688-4468-a52c-c3e3d9b437bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.570454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-fe70fc1b-d688-4468-a52c-c3e3d9b437bd None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:50.571654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fe70fc1b-d688-4468-a52c-c3e3d9b437bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:50.650341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 409/808] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:50.651011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70fc1b-d688-4468-a52c-c3e3d9b437bd None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:50.668611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d37f8-6854-4e07-8c64-bc780f4d04f0 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:50.684321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:50.715423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:50.723987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:50.724721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.725004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.725686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.726500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:50.726936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:50.729129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:50.745261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 400/809] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:50.746121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:50.799301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70fc1b-d688-4468-a52c-c3e3d9b437bd None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:50.845689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:50.846332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.847135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.847412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.860830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:50.867712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:50.868651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:50.869462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:50.872304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 410/810] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:50.873135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:50.880533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ba7af-6941-4dc8-b4df-205a80cd50d5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:50.920945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:50.921583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:50.921862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:50.922174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:50.922919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:50.924482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:50.934511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:50.935700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:50.936786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 401/811] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:50 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:50.937775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:50.982441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.010973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.013439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df96958-8316-4ce3-bea2-8e49b9ee5eec None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.062042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e2d66-46b4-426d-b2e4-7191fdc035a9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.063422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.064509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.065841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.066530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.075979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.078988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:51.095857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:51.097067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:42:51.098073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:51.099596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.100268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 411/812] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:42:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:51.101188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.104964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.106043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.106043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.106264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:51.106636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.108038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.123186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 402/813] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:51.123186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.234632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919055cf-a363-4c4a-a0f9-21677ed084af None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.239632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22b5b4-7801-4bfa-aa1b-c43f32bec995 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.280528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.283557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.284202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.284645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.284926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.285557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] RBAC: Adding query filter params (name=network) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:51.285862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.287229 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.288756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.289662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.290121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.290588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.291590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:51.292124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.293823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.297666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 412/814] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:51.298398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.302010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.316922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 403/815] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:51.318058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.411440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2cbdb-81bd-43aa-b781-1ba80944c57e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.428614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09787315-8f28-4777-9c5d-35d99c4de15b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.453409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.454048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.454380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.454600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.465405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.466846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.469333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.470188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.470728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.471215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.472041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] RBAC: Adding query filter params (type=network) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:51.472714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.474435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.481307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 404/816] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:51.482354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:51.486437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:51.488297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:51.488297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:51.488297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:51.488297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 413/817] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:51.488963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.532536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.551177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.598566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbaeff-edd9-4181-9b1c-0963ef031d51 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.615064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f945-e31a-4ef6-82ae-242ef054063b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.640492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.641296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.641653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.644107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.644107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.644107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.666674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.667317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.667600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.667874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.672906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 414/818] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:42:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:51.673951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.681302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.684275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:51.701514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:51.702545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:51.705665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:51.705665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:51.705665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:51.705665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 405/819] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:51.712923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.827636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.840560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19839a-7f1c-4d8a-acde-967c7c040f7b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.891477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.892522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.893012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.894060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.894945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:51.895472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.897228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.912991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 406/820] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:51.919425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:51.937552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc078fe-d1b6-4e17-a38b-d7a9418735a3 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:51.952276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:51.979851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:51.980507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:51.980790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:51.981239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:51.981939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:51.982257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:51.983635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:51.998908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 415/821] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:52.005036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.078401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e5b1-cd2d-47d6-ac4c-0beb9b1a03d5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.128833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.129845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.130332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.131362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.142197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:52.143664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:52.149390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ae50-13d9-4c93-a706-851665a7c007 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:52.155389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:52.156922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:52.157802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:52.162581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 407/822] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:52.163514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.172418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:52.199162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7b751-a392-44de-b3a9-4cd551457230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.200788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7b751-a392-44de-b3a9-4cd551457230 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.201288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7b751-a392-44de-b3a9-4cd551457230 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.201909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7b751-a392-44de-b3a9-4cd551457230 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.212896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7b751-a392-44de-b3a9-4cd551457230 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:52.214345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7b751-a392-44de-b3a9-4cd551457230 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-51b7b751-a392-44de-b3a9-4cd551457230 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:52.230678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:52.231644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:52.232970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 416/823] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:52.233545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d370a24-2d3d-4e2c-9ae2-00c94472518f None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.233545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d370a24-2d3d-4e2c-9ae2-00c94472518f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.233545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d370a24-2d3d-4e2c-9ae2-00c94472518f None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.238877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 417/824] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:52.238877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d370a24-2d3d-4e2c-9ae2-00c94472518f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.271716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:52.294615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52b6ce-9075-4b35-ae0e-bcefb4b1bb59 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.331849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a7aaa-9619-4dac-86e5-967f4584bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.332406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a7aaa-9619-4dac-86e5-967f4584bf6b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.332739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a7aaa-9619-4dac-86e5-967f4584bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.333012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a7aaa-9619-4dac-86e5-967f4584bf6b None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.333700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a7aaa-9619-4dac-86e5-967f4584bf6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:52.334003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a7aaa-9619-4dac-86e5-967f4584bf6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:52.335440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a7aaa-9619-4dac-86e5-967f4584bf6b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:52.353148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 408/825] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:52.353148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bfb24-d40b-4b20-abfb-f0fe2e51d2a9 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.353148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bfb24-d40b-4b20-abfb-f0fe2e51d2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.353148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bfb24-d40b-4b20-abfb-f0fe2e51d2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.354478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d370a24-2d3d-4e2c-9ae2-00c94472518f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.371000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-426bfb24-d40b-4b20-abfb-f0fe2e51d2a9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:52.372220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-426bfb24-d40b-4b20-abfb-f0fe2e51d2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:52.389800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.390413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.391303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.391572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.392329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:52.392643 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:52.394059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:52.418936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 418/826] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:52.418936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.454642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:52.463472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 409/827] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:52.473552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bfb24-d40b-4b20-abfb-f0fe2e51d2a9 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.519525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:52.531313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db05ff4-06b8-4785-8cb4-0e6a80787bba None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.567474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.568175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.568466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.568742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/81fba04097094820be201ad24ed2658d/roles/6aaf096343b84cfea3eb6e8fd11ba645` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.584804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=81fba04097094820be201ad24ed2658d, role_id=6aaf096343b84cfea3eb6e8fd11ba645)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:52.585964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bfb24-d40b-4b20-abfb-f0fe2e51d2a9 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.586205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:52.600105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 419/828] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:42:52 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/81fba04097094820be201ad24ed2658d/roles/6aaf096343b84cfea3eb6e8fd11ba645 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:42:52.601250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.635023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b8f60-5f02-43cb-8b32-fdfd510aac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.636059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b8f60-5f02-43cb-8b32-fdfd510aac03 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.636400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b8f60-5f02-43cb-8b32-fdfd510aac03 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.636961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b8f60-5f02-43cb-8b32-fdfd510aac03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.717268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:52.721724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31315b33-3c0f-40fc-8c52-3b607a8f4ce6 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.726647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b8f60-5f02-43cb-8b32-fdfd510aac03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:52.728121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b8f60-5f02-43cb-8b32-fdfd510aac03 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:52.765277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9033fd-4523-4958-8f8b-e73a71baf1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.766486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9033fd-4523-4958-8f8b-e73a71baf1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.766769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9033fd-4523-4958-8f8b-e73a71baf1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.767036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9033fd-4523-4958-8f8b-e73a71baf1f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.777057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:52.784188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9033fd-4523-4958-8f8b-e73a71baf1f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-6a9033fd-4523-4958-8f8b-e73a71baf1f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:52.787196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:52.788095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:52.789610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 420/829] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:52.790303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc70a7-a808-4b27-a32d-88327e7517ac None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.790303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc70a7-a808-4b27-a32d-88327e7517ac None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.791134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc70a7-a808-4b27-a32d-88327e7517ac None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.796195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 421/830] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:52.796195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc70a7-a808-4b27-a32d-88327e7517ac None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.849775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 410/831] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:42:52.851008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab3ae-7cd0-4c11-8c4b-9dafbebddb73 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.851296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab3ae-7cd0-4c11-8c4b-9dafbebddb73 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.851570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab3ae-7cd0-4c11-8c4b-9dafbebddb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.868939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ea3ab3ae-7cd0-4c11-8c4b-9dafbebddb73 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:52.870222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ea3ab3ae-7cd0-4c11-8c4b-9dafbebddb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:52.907432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc70a7-a808-4b27-a32d-88327e7517ac None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:52.948628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 411/832] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:52.948628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab3ae-7cd0-4c11-8c4b-9dafbebddb73 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.948877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:52.948877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:52.948877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:52.948877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:52.949461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:52.949461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:52.949461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:52.959642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 422/833] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:52.965207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:52.981151 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.028428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.071969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e162ad1-fa51-427a-8d11-4fdabb607983 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.075821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab3ae-7cd0-4c11-8c4b-9dafbebddb73 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.109234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.110620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.111061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.111330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.118446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.119144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.119497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.119835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.122707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.124174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.131309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.132846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:53.139641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:53.140594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:53.141523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:53.141523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:53.141523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:53.141523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 423/834] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:53.143503 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:53.150542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:53.151462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:53.152368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:53.152368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:53.152368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:53.152368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 412/835] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:53.152993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.235241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.259494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb45cd-0506-4003-a4e2-71db84e968c4 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.266457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9a210-c8b7-406d-9d99-d54318027da0 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.286185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.295393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.295945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.296329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.296628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.297873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:53.298170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.299530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.311871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 413/836] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:53.312726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.317420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.318069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.318352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.318626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.319510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.321484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:53.330529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:53.331526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:53.332398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 424/837] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:53.333179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.432416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c975a-4bdc-4b91-8623-ba46b04a095e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.449557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f58f0-2f58-4dad-93c4-632a8aafc63c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.467393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.468138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.468400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.468681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.469902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:53.470210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.471631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.478380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.484298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 414/838] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:53.485665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.491844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.492517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.492808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.493082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.500580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.502021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:53.514104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:53.515016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 425/839] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:53.516428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.563045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.599850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930452-d6cf-4176-99cb-91fbc8d51012 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.630382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881abb4-9190-4ba1-83db-0bbf84d46992 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.636591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.637206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.637494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.637767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.638394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:53.638699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.640258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.645411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 415/840] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:53.646201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.677245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.677898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.678331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.678619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.687316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.689240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:53.705727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:53.706682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:53.707560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 426/841] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:53.713705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.714840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.755307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef028d9a-32b5-41ec-b7f3-ad0b6a12273f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.791918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.792558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.792835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.793105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.794275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:53.794669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.796554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.814568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 416/842] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:53.815363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.821972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.826689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9802bc-0072-44a1-91bb-75ec01babd31 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.864821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.865434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.865713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.865984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.866609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:53.866914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.868308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.873308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 427/843] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:53.883408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:53.923865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348cafb-fd63-4012-8f95-e06fc4447e66 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:53.948532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:53.967428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:53.968050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:53.969500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:53.969671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:53.970436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:53.970745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:53.972366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:53.990746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 417/844] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:53.991987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:54.036009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bad7a6-6214-42fb-9ac2-f5e7592a5920 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:54.086176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:54.087059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.088291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.088590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.120581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e211ebe-35c8-4ec6-9790-f58407d4a99b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:54.128625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:54.161910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:54.163017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.163865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.164589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.165710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=86456c6dbe4e454e9f255960b70c0832) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:54.166165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:54.168448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:54.177602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 418/845] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/role_assignments?user.id=86456c6dbe4e454e9f255960b70c0832&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:54.177782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:54.179189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:54.304164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 428/846] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:42:54.310036 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:54.362222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:54.426882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a040d-0d53-4fba-9488-1b2a59a4327e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:54.461673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2f047-d908-4832-b8ea-caa09051980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:54.462455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2f047-d908-4832-b8ea-caa09051980f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.462740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2f047-d908-4832-b8ea-caa09051980f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.463017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2f047-d908-4832-b8ea-caa09051980f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.476686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2f047-d908-4832-b8ea-caa09051980f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:54.478165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2f047-d908-4832-b8ea-caa09051980f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d9b2f047-d908-4832-b8ea-caa09051980f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:54.497733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:54.498800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:54.499670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:54.499670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:54.499670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:54.499670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 419/847] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:54.504834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:54.571546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:54.615000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc281b-4840-44c0-95ef-75a384c12578 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:54.625650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aad050-d61e-44fd-96d0-601a9ee236b1 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.625945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aad050-d61e-44fd-96d0-601a9ee236b1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.626206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aad050-d61e-44fd-96d0-601a9ee236b1 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.627805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 420/848] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:54.632624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48a00-675c-4679-94d9-490b9828eb94 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.632909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48a00-675c-4679-94d9-490b9828eb94 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.633184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48a00-675c-4679-94d9-490b9828eb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.651988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-75d48a00-675c-4679-94d9-490b9828eb94 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:54.653071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:54.653275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-75d48a00-675c-4679-94d9-490b9828eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:54.653687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.653968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.654242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.654894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:54.655210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:54.657170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:54.669615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 429/849] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:54.675104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:54.744106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 421/850] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:42:54.768741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48a00-675c-4679-94d9-490b9828eb94 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:54.788565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124370-a3ac-4a65-a810-0111f9284a06 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:54.804174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:54.817986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:54.833730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:54.834323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.834556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.834827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.843242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:54.845268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:54.859002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:54.859935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:54.860848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 430/851] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:54.866959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:54.899135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48a00-675c-4679-94d9-490b9828eb94 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:54.947197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:54.947838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:54.948281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:54.948583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:54.962895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:54.969439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:54.970371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:54.971196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 422/852] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:42:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:54.976880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:54.990708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd34a4-25af-48c9-89f2-019a81a9b032 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:55.030520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:55.031561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.032048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.032553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.033600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:55.034118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:55.035904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:55.055760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 431/853] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:55.057203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:55.091302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:55.109961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b250cb-0fc3-4822-bdfb-711a7aa18506 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:55.120548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:55.154507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:55.155119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.155402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.155674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.156523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:55.157048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:55.158442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:55.172532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 423/854] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:42:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:55.175573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:55.181940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ea17b-2adb-406b-bc37-48a1959023a5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:55.216842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:55.217541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.217826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.218098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.218843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=f64c104787604ea4b22e2af2507b3f0a) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:55.219152 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:55.220614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:55.228402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 432/855] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:55 2022] GET /identity/v3/role_assignments?user.id=f64c104787604ea4b22e2af2507b3f0a&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:55.295912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d01d32-9bbb-448f-8105-e824e172e094 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:55.334751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:55.335803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.336334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.336810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.349311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:55.351223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:55.361518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:55.368367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:55.369346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:55.370708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:55.370708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:55.370708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:55.370708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 424/856] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:55.376025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:55.488426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10fcc5-3099-48e7-8058-d1ad474884f1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:55.496208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:55.529373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:55.530464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.530954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.532013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.532956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:55.533466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:55.535062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:55.548293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 433/857] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:55.556187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:55.658220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fa37e-b531-46f6-9840-5a7b26d0c8e6 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:55.693330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:55.693963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.694244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.694517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.702261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:55.704406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:42:55.715839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:55.716831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:55.717794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 425/858] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:42:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:55.723035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:55.834670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed31f1-665e-42f0-bf6a-b2cf985734d1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:55.870304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66a805-ed14-4408-b246-e69a3de07f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:55.870963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66a805-ed14-4408-b246-e69a3de07f9a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.871245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66a805-ed14-4408-b246-e69a3de07f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.871514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66a805-ed14-4408-b246-e69a3de07f9a None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.872262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66a805-ed14-4408-b246-e69a3de07f9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:55.872568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66a805-ed14-4408-b246-e69a3de07f9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:55.874036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66a805-ed14-4408-b246-e69a3de07f9a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:55.893628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 434/859] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 10 16:42:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:55.899068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:55.964786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:55.970886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114470d6-01bc-4513-a52b-04cf726e2450 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.971478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114470d6-01bc-4513-a52b-04cf726e2450 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.971954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114470d6-01bc-4513-a52b-04cf726e2450 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.973865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 435/860] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:55.979962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0757e3-56b1-4933-9035-4725d6b883f9 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:55.980325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0757e3-56b1-4933-9035-4725d6b883f9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:55.980601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0757e3-56b1-4933-9035-4725d6b883f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:55.999591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-bf0757e3-56b1-4933-9035-4725d6b883f9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:56.002453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bf0757e3-56b1-4933-9035-4725d6b883f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:56.010317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e431e4-6e95-473e-a4ec-b7bbb93657bc None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:56.031064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:56.048206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:56.048877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.049158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.049429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.088298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 436/861] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:42:56.105646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0757e3-56b1-4933-9035-4725d6b883f9 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.120797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:56.122250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:56.182956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:56.220487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0757e3-56b1-4933-9035-4725d6b883f9 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:56.226819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 426/862] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:42:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:42:56.233620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.268975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75639efe-5ba9-4f24-af9f-621f794241ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:56.269597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639efe-5ba9-4f24-af9f-621f794241ca None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.269875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639efe-5ba9-4f24-af9f-621f794241ca None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.270144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639efe-5ba9-4f24-af9f-621f794241ca None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.271073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75639efe-5ba9-4f24-af9f-621f794241ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:56.272626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75639efe-5ba9-4f24-af9f-621f794241ca None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:56.294024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 437/863] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:42:56 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:42:56.365748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9152c-d474-4676-ae0b-5044ffafb9d5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:56.370449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1caee0-284d-41e5-a187-ef375f8a82d3 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.370762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1caee0-284d-41e5-a187-ef375f8a82d3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.371010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1caee0-284d-41e5-a187-ef375f8a82d3 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 438/864] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:56.378303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472ffdb-983d-4df9-b54d-91540c5dc40e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.378619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472ffdb-983d-4df9-b54d-91540c5dc40e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.378986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472ffdb-983d-4df9-b54d-91540c5dc40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.397917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a472ffdb-983d-4df9-b54d-91540c5dc40e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:56.399143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a472ffdb-983d-4df9-b54d-91540c5dc40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:56.420098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:56.420920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.421199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.421471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.428591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:56.436763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:56.438955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:42:56.460317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:56.461276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:42:56.462136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:42:56.462136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:42:56.462136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:56.462136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 427/865] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:56.467704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.483740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:56.499144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 439/866] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:42:56.522866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472ffdb-983d-4df9-b54d-91540c5dc40e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.596044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f95043-e3e0-4727-a8fb-4e6acf29cdd2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:56.633756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:56.634365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.634642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.634910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.635664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:56.635962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:56.637527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:56.647913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472ffdb-983d-4df9-b54d-91540c5dc40e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:56.657293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 428/867] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:56.659135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.690913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:56.691742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.692018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.692322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.692902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:56.694484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:56.704275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:56.705200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:42:56.706100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:56.710479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 440/868] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Aug 10 16:42:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:56.711780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.725423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:56.745371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:56.791091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516d270-69e6-4550-ad15-2dbcb9bcf93c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:56.832383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46abc7-03df-45ae-bfe6-5653e97428e5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:56.834811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:56.836350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.836701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.837040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.846154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:56.847864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:42:56.864251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:56.865340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:42:56.866399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:56.871692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 429/869] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:56.873155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.877047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:56.877662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:56.878756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:56.879103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:56.879741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:56.880101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:56.881493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:56.892326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 441/870] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:42:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:56.893967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:56.969829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:57.001960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389739f-f5fd-4066-b522-aae7120db99a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:57.003406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6424c65-40c6-4bd2-bb10-45d1bef46dc2 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:57.035488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:57.045479 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c61a53-24d0-45a4-ae9b-eb5dd087c41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:57.046503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c61a53-24d0-45a4-ae9b-eb5dd087c41c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:57.046962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c61a53-24d0-45a4-ae9b-eb5dd087c41c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:57.047375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c61a53-24d0-45a4-ae9b-eb5dd087c41c None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:57.048328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:57.048492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c61a53-24d0-45a4-ae9b-eb5dd087c41c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:57.048956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:57.049203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c61a53-24d0-45a4-ae9b-eb5dd087c41c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:57.049305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:57.049481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:57.050107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:57.050406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:57.051295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c61a53-24d0-45a4-ae9b-eb5dd087c41c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:57.051757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:57.060190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 442/871] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:42:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:57.080293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 430/872] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:42:57.082040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:57.198909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c590330-6f3c-45cf-a3ad-ba7b62de1155 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:57.236715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cbb35-52ca-4c6f-9c3d-68885daedfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:57.237577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cbb35-52ca-4c6f-9c3d-68885daedfab None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:57.237855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cbb35-52ca-4c6f-9c3d-68885daedfab None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:57.238124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cbb35-52ca-4c6f-9c3d-68885daedfab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:57.238842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cbb35-52ca-4c6f-9c3d-68885daedfab None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=81fba04097094820be201ad24ed2658d) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:57.239148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cbb35-52ca-4c6f-9c3d-68885daedfab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:57.240616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cbb35-52ca-4c6f-9c3d-68885daedfab None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:57.244000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:57.248180 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 443/873] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:42:57 2022] GET /identity/v3/role_assignments?user.id=81fba04097094820be201ad24ed2658d&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:57.428305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa5338-4d4b-4ee7-bf42-b45ac64bcc1d None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:57.428941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa5338-4d4b-4ee7-bf42-b45ac64bcc1d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:57.429408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa5338-4d4b-4ee7-bf42-b45ac64bcc1d None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:57.430889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 431/874] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:57.436549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a8108-88e1-432b-89f5-364c738f745c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:57.437033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a8108-88e1-432b-89f5-364c738f745c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:57.438105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a8108-88e1-432b-89f5-364c738f745c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:57.461522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c04a8108-88e1-432b-89f5-364c738f745c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:57.462719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c04a8108-88e1-432b-89f5-364c738f745c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:57.536252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 444/875] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:42:57.558465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa5338-4d4b-4ee7-bf42-b45ac64bcc1d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:57.668664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa5338-4d4b-4ee7-bf42-b45ac64bcc1d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:57.705109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:57.705740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:57.706016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:57.706285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:57.706859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:57.708235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:57.717512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:57.718491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:57.719769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 432/876] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:42:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:42:57.727572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a8108-88e1-432b-89f5-364c738f745c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:57.831897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a8108-88e1-432b-89f5-364c738f745c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:57.863319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:57.871156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbded1-3b1c-4ea0-8e86-f0389c80acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:57.871777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbded1-3b1c-4ea0-8e86-f0389c80acee None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:57.872060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbded1-3b1c-4ea0-8e86-f0389c80acee None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:57.872366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbded1-3b1c-4ea0-8e86-f0389c80acee None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:57.873010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbded1-3b1c-4ea0-8e86-f0389c80acee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:57.873317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbded1-3b1c-4ea0-8e86-f0389c80acee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:57.874689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbded1-3b1c-4ea0-8e86-f0389c80acee None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:57.879957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 445/877] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:42:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:57.885944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:57.942209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:57.991722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d18bc-9aa8-479c-85a6-f6da909cedcb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:58.027174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ed3-f3ea-494e-a783-f350d4f25721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:58.027782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ed3-f3ea-494e-a783-f350d4f25721 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:58.028059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ed3-f3ea-494e-a783-f350d4f25721 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:58.028353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ed3-f3ea-494e-a783-f350d4f25721 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:58.028976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ed3-f3ea-494e-a783-f350d4f25721 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:58.029275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ed3-f3ea-494e-a783-f350d4f25721 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:58.030896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ed3-f3ea-494e-a783-f350d4f25721 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:58.037598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 433/878] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:42:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:58.397366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aa89b-b115-4c27-94a3-96c33d06d378 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:58.397744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aa89b-b115-4c27-94a3-96c33d06d378 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:58.397921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aa89b-b115-4c27-94a3-96c33d06d378 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:58.399423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 446/879] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:58.404950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c26eb5-9dd8-4a48-ab74-733476d46315 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:58.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c26eb5-9dd8-4a48-ab74-733476d46315 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:58.405508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c26eb5-9dd8-4a48-ab74-733476d46315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:58.430328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e9c26eb5-9dd8-4a48-ab74-733476d46315 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:58.431835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e9c26eb5-9dd8-4a48-ab74-733476d46315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:58.506345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 434/880] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:58 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:42:58.530763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aa89b-b115-4c27-94a3-96c33d06d378 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:58.640234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aa89b-b115-4c27-94a3-96c33d06d378 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:58.676761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:58.677405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:58.677687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:58.677960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:58.678546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:58.679916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:58.689569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:58.690521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:42:58.692243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 447/881] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:58.696934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c26eb5-9dd8-4a48-ab74-733476d46315 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:58.746439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:58.805228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c26eb5-9dd8-4a48-ab74-733476d46315 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:58.842030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:58.842652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:58.842931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:58.843206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:58.843875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:58.844199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:58.845591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:58.850971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 435/882] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:58.856579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:58.919709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:58.969191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6407843-d6d5-4863-ad76-f51f6c2b0e74 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:59.005610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376eff9-ff0c-4df8-be94-c597d922e3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:59.006228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376eff9-ff0c-4df8-be94-c597d922e3de None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.006512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376eff9-ff0c-4df8-be94-c597d922e3de None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.006894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376eff9-ff0c-4df8-be94-c597d922e3de None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.007535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376eff9-ff0c-4df8-be94-c597d922e3de None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:59.007842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376eff9-ff0c-4df8-be94-c597d922e3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:59.009253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376eff9-ff0c-4df8-be94-c597d922e3de None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:59.014805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 448/883] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:42:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:59.020392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:59.126011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9deb-6067-49f3-8871-5c000268e268 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:59.137641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:59.162842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:59.163457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd16f3-bc85-4a6f-aa05-8076629cc516 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.163571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.163754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.163846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd16f3-bc85-4a6f-aa05-8076629cc516 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.164000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.164089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd16f3-bc85-4a6f-aa05-8076629cc516 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.164731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] RBAC: Adding query filter params (interface=public, service_id=884490607fb44b3b9fd4ca5c9a59b64f, region_id=RegionOne) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:42:59.165037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:59.165631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 449/884] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:42:59.166403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:59.170756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a08bb-c591-4eb3-952e-6956d25d67b2 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.172184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a08bb-c591-4eb3-952e-6956d25d67b2 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.172184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a08bb-c591-4eb3-952e-6956d25d67b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.173070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 436/885] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:42:59 2022] GET /identity/v3/endpoints?service_id=884490607fb44b3b9fd4ca5c9a59b64f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:42:59.179198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:59.194973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-081a08bb-c591-4eb3-952e-6956d25d67b2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:59.196265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-081a08bb-c591-4eb3-952e-6956d25d67b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:59.285179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 450/886] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:42:59.298886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef2482-e63d-4c4b-b3d1-b32e50ae5552 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:59.311309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a08bb-c591-4eb3-952e-6956d25d67b2 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:59.342238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948a3de-d0a9-4f79-829d-1b252cb85135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:59.342987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948a3de-d0a9-4f79-829d-1b252cb85135 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.343306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948a3de-d0a9-4f79-829d-1b252cb85135 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.343585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948a3de-d0a9-4f79-829d-1b252cb85135 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.344256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948a3de-d0a9-4f79-829d-1b252cb85135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:59.345748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948a3de-d0a9-4f79-829d-1b252cb85135 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:59.351903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 437/887] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:42:59 2022] GET /identity/v3/services => generated 1011 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:42:59.371191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:59.437256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a08bb-c591-4eb3-952e-6956d25d67b2 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:59.476727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8879e8b-94f8-4918-a653-dbcd42567e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:59.477727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8879e8b-94f8-4918-a653-dbcd42567e51 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.478154 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8879e8b-94f8-4918-a653-dbcd42567e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.478587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8879e8b-94f8-4918-a653-dbcd42567e51 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.479396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8879e8b-94f8-4918-a653-dbcd42567e51 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:42:59.480949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8879e8b-94f8-4918-a653-dbcd42567e51 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:42:59.501466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 451/888] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:42:59 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:42:59.525110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5f6f-d7e7-49f7-902b-4d435d0019d0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.525538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5f6f-d7e7-49f7-902b-4d435d0019d0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.525950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5f6f-d7e7-49f7-902b-4d435d0019d0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.529198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 438/889] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:42:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:42:59.533799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf562ce-2955-4f84-b1f9-db598dd41552 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.534092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf562ce-2955-4f84-b1f9-db598dd41552 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.534374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf562ce-2955-4f84-b1f9-db598dd41552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.554469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-dcf562ce-2955-4f84-b1f9-db598dd41552 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:42:59.555632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-dcf562ce-2955-4f84-b1f9-db598dd41552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:42:59.640184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 452/890] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:42:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:42:59.664180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a5f6f-d7e7-49f7-902b-4d435d0019d0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:59.710481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:59.782166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a5f6f-d7e7-49f7-902b-4d435d0019d0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:59.818647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:59.819250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.819529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.819806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:42:59.831549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:42:59.837578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:42:59.838553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:42:59.839381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:42:59.841304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 439/891] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 10 16:42:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:42:59.846725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf562ce-2955-4f84-b1f9-db598dd41552 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:42:59.893946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:42:59.961363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf562ce-2955-4f84-b1f9-db598dd41552 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:42:59.998161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59799ba2-d988-462a-a5e0-0e3fedebe2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:42:59.998780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59799ba2-d988-462a-a5e0-0e3fedebe2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:42:59.999062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59799ba2-d988-462a-a5e0-0e3fedebe2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:42:59.999338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59799ba2-d988-462a-a5e0-0e3fedebe2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.000586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59799ba2-d988-462a-a5e0-0e3fedebe2d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:00.000899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59799ba2-d988-462a-a5e0-0e3fedebe2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:00.002261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59799ba2-d988-462a-a5e0-0e3fedebe2d0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:00.010839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 453/892] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 10 16:42:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:00.016038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:00.137194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:00.141677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50dde6a-d501-4c65-918a-ff2ec6c0b284 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:00.147081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ac1df-bda9-482c-8b26-1e3c6de261f4 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:00.147373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ac1df-bda9-482c-8b26-1e3c6de261f4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:00.147653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ac1df-bda9-482c-8b26-1e3c6de261f4 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.149270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 454/893] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:00.158243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5f642-b262-4521-836d-fd458ce6d93b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:00.159235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5f642-b262-4521-836d-fd458ce6d93b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:00.159672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5f642-b262-4521-836d-fd458ce6d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.185887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8bb5f642-b262-4521-836d-fd458ce6d93b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:00.187116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8bb5f642-b262-4521-836d-fd458ce6d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:00.193146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:00.194175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:00.194656 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:00.195124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.207054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:00.209181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:00.226361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:00.227533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:00.228421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:00.228421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:00.228421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:00.230561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 440/894] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:00.237109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:00.269966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 455/895] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:00.292590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5f642-b262-4521-836d-fd458ce6d93b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:00.324404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:00.365657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db78b90-e1da-4427-99d4-d51b4fa0fec9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:00.405759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5f642-b262-4521-836d-fd458ce6d93b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:00.411311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:00.411937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:00.412246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:00.412523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.413179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:00.413484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:00.414865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:00.429655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 441/896] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:00.441424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:00.454062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:00.460200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:00.460996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:00.461422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:00.461961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.462853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:00.464438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:00.496623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 456/897] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:43:00 2022] POST /identity/v3/services => generated 241 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:00.565247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf0cb1-6c28-4b13-a51c-440c2db7ef75 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:00.623272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:00.624352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:00.624837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:00.625340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.634646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:00.636629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:43:00.651595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:00.652627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:43:00.653560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:00.656902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 442/898] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:43:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:00.665005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:00.763091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:00.775091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd89d6-b458-4855-b1e5-d3045d4dc41d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:00.817101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:00.818179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:00.818661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:00.819125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:00.820058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:00.820585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:00.822305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:00.841821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 457/899] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 10 16:43:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:00.848738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:00.918573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:00.964380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5634a7-4b54-46b6-8c60-868daa899e7c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:01.001119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:01.002048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.002514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.002986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.077397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:01.079767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:01.148937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:01.190611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 443/900] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:01.197308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:01.304796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e945c4-4036-4d9b-9d1f-c6d928cedb86 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:01.340415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06261678-bb58-4771-a107-6a8cc744821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:01.341055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261678-bb58-4771-a107-6a8cc744821d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.341336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261678-bb58-4771-a107-6a8cc744821d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.341603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261678-bb58-4771-a107-6a8cc744821d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.354353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06261678-bb58-4771-a107-6a8cc744821d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:01.356818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06261678-bb58-4771-a107-6a8cc744821d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:01.371415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-06261678-bb58-4771-a107-6a8cc744821d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:01.378486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:01.379489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:01.380493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:01.380493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:01.380493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:01.380493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 458/901] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:01.385673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:01.487485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50179b13-4d66-406d-bbe5-13ed5f408b32 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.487693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50179b13-4d66-406d-bbe5-13ed5f408b32 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.487950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50179b13-4d66-406d-bbe5-13ed5f408b32 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.493302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 459/902] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:01.495170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2546-ce41-4aef-b18a-678576f8d642 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.495550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2546-ce41-4aef-b18a-678576f8d642 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.495900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2546-ce41-4aef-b18a-678576f8d642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.501554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86982ee-4b8d-4145-9d4f-333067a014d0 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:01.515039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5bab2546-ce41-4aef-b18a-678576f8d642 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:01.516393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5bab2546-ce41-4aef-b18a-678576f8d642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:01.543961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:01.544920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.545205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.545746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.546394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:01.546724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:01.548945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:01.562360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 444/903] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:01.567527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:01.598464 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:01.604921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 460/904] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:01.629439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab2546-ce41-4aef-b18a-678576f8d642 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:01.681211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686472ff-2a90-4100-a6fb-d30dea9f066e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:01.719467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c151e5-fa7a-42ee-8ac7-ea803acff058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:01.720264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c151e5-fa7a-42ee-8ac7-ea803acff058 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.720564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c151e5-fa7a-42ee-8ac7-ea803acff058 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.720918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c151e5-fa7a-42ee-8ac7-ea803acff058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.728362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c151e5-fa7a-42ee-8ac7-ea803acff058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:01.729951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c151e5-fa7a-42ee-8ac7-ea803acff058 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-12c151e5-fa7a-42ee-8ac7-ea803acff058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:43:01.744308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:01.745224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:01.746111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:01.747061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 445/905] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:01.748576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725951d3-1b3d-42e2-a86e-b49007a21de8 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.749073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725951d3-1b3d-42e2-a86e-b49007a21de8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.749573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725951d3-1b3d-42e2-a86e-b49007a21de8 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.751105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 446/906] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:01.752024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725951d3-1b3d-42e2-a86e-b49007a21de8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:01.759137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab2546-ce41-4aef-b18a-678576f8d642 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:01.792065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:01.799825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f8800-4bcb-4cb1-bf62-6403053bad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:01.800508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8800-4bcb-4cb1-bf62-6403053bad79 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.800789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8800-4bcb-4cb1-bf62-6403053bad79 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.801064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8800-4bcb-4cb1-bf62-6403053bad79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.801658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f8800-4bcb-4cb1-bf62-6403053bad79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:01.803044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f8800-4bcb-4cb1-bf62-6403053bad79 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-215f8800-4bcb-4cb1-bf62-6403053bad79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:01.812381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:01.818768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:01.819614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 461/907] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:43:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:01.820423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a82da7-9d94-4b73-b071-0b2993bf8e3e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.820423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a82da7-9d94-4b73-b071-0b2993bf8e3e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.820423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a82da7-9d94-4b73-b071-0b2993bf8e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.837562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-15a82da7-9d94-4b73-b071-0b2993bf8e3e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:01.838774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-15a82da7-9d94-4b73-b071-0b2993bf8e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:01.861599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:01.879017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725951d3-1b3d-42e2-a86e-b49007a21de8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:01.913933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 462/908] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:01.920349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a82da7-9d94-4b73-b071-0b2993bf8e3e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:01.925916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:01.926539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:01.926820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:01.927098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:01.927796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:01.928102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:01.929571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:01.952027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 447/909] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:01.953906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.026142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a82da7-9d94-4b73-b071-0b2993bf8e3e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.051723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:02.063386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.064061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.064194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f322d-2b7c-44ce-ae7c-dbe22649557b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.064379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.064658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.065291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:02.065602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.067233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.071904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 463/910] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:43:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:02.072716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.109079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.110094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.110223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.110545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.111132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.112532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:02.121615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:02.122569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:02.123445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 448/911] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Aug 10 16:43:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:02.124291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.138818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:02.189623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22680-1834-47d2-b8a3-e0c4e1d07267 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.225377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.226006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.226261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.226538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.227266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] RBAC: Adding query filter params (role.id=457c69fdf6cc4e33885c6cc4bac6b9ec, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=81fba04097094820be201ad24ed2658d) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:02.227577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.229039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.244173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 464/912] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:02 2022] GET /identity/v3/role_assignments?user.id=81fba04097094820be201ad24ed2658d&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=457c69fdf6cc4e33885c6cc4bac6b9ec => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:02.244173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.244505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb1d22-1a73-4695-80dd-42e652d02b8b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.298722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.299707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.300055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.300422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.301296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:02.301664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.303381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.308518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:02.312932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 449/913] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 10 16:43:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:02.319983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.380088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24dbe1-ca3b-4010-a3fa-7b94ca8463cc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.420342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.421028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.421305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.421635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.422317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:02.422585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.424013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.424373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:02.432258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 465/914] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:02.435487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.465507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d896f-4fe8-4b48-a966-2beb6a2fbddb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.518080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.518697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.518977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.519660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.520612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.521998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.556282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 450/915] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:43:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:43:02.571708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f092573-f386-4fbe-83c4-89732c90afcb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.593861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:02.623254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88665a1-f933-4146-b3cd-83c51be9c1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.624182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88665a1-f933-4146-b3cd-83c51be9c1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.625445 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88665a1-f933-4146-b3cd-83c51be9c1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.625972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88665a1-f933-4146-b3cd-83c51be9c1b7 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.626787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88665a1-f933-4146-b3cd-83c51be9c1b7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:02.627297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88665a1-f933-4146-b3cd-83c51be9c1b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.628960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88665a1-f933-4146-b3cd-83c51be9c1b7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.635526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 466/916] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:02.642989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.707374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a43b0-feaf-4c0b-9d75-a160ffaac266 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.707713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a43b0-feaf-4c0b-9d75-a160ffaac266 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.707928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a43b0-feaf-4c0b-9d75-a160ffaac266 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.709601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 467/917] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:02.715962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956027f-89f2-4046-b02a-cac9ef1e5999 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.716627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956027f-89f2-4046-b02a-cac9ef1e5999 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.716919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956027f-89f2-4046-b02a-cac9ef1e5999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.739868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e956027f-89f2-4046-b02a-cac9ef1e5999 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:02.741113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e956027f-89f2-4046-b02a-cac9ef1e5999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:02.773029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a27b6-a8d5-4d5e-8be8-d1cf1606363e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.809591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:02.820237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:02.829153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:02.829842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:02.830337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:02.831256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] RBAC: Adding query filter params (interface=public, service_id=b5767165a2a143f68f25cdf52b4f2100, region_id=RegionOne) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:02.831763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:02.833569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:02.834818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 468/918] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:02.840268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 451/919] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:43:02 2022] GET /identity/v3/endpoints?service_id=b5767165a2a143f68f25cdf52b4f2100&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:02.846136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956027f-89f2-4046-b02a-cac9ef1e5999 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.867534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:02.975473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb3844-fb4f-4bd0-9e4c-b4851fff260b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.979720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956027f-89f2-4046-b02a-cac9ef1e5999 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:02.992586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:03.012861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:03.013663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:03.014040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:03.014472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:03.015157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:03.016827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:03.024879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:03.025592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:03.025879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:03.026221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:03.026526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:03.027645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:03.028686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:03.029584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:03.029584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:03.032185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 452/920] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:03.037452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:03.037835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 469/921] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:43:02 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:43:03.128673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:03.158924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845294ab-c894-4fd3-aaae-cb11ef5a1cdc None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:03.203220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:03.203832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:03.204109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:03.204487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:03.205121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:03.205422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:03.206797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:03.216220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 453/922] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:03.217912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:03.322233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606b709-06c8-40ba-8497-43f9c090f3e8 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:03.357534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:03.358151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:03.358484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:03.358763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:03.360011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:03.360408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:03.361784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:03.367249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 470/923] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:03.373102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:03.489832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f6a15-d956-4bf1-97a2-30b2254e9125 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:03.512826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:03.525744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49731b3-dae9-4724-9a10-05ccd06e01ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:03.526355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49731b3-dae9-4724-9a10-05ccd06e01ba None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:03.526633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49731b3-dae9-4724-9a10-05ccd06e01ba None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:03.526907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49731b3-dae9-4724-9a10-05ccd06e01ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:03.527601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49731b3-dae9-4724-9a10-05ccd06e01ba None admin] RBAC: Adding query filter params (interface=public, service_id=08dabb19ccc24c7bbf2b3dc06f951b93, region_id=RegionOne) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:03.527903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49731b3-dae9-4724-9a10-05ccd06e01ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:03.529301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49731b3-dae9-4724-9a10-05ccd06e01ba None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:03.535572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 454/924] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:43:03 2022] GET /identity/v3/endpoints?service_id=08dabb19ccc24c7bbf2b3dc06f951b93&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:03.541068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:03.575870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:03.645226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19ba9e-c460-4b03-8005-596a57f5a095 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:03.679426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b50b-b65c-4252-9ddf-34ce5729f4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:03.680046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b50b-b65c-4252-9ddf-34ce5729f4bf None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:03.680350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b50b-b65c-4252-9ddf-34ce5729f4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:03.680620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b50b-b65c-4252-9ddf-34ce5729f4bf None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:03.681216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b50b-b65c-4252-9ddf-34ce5729f4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:03.682581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b50b-b65c-4252-9ddf-34ce5729f4bf None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:03.689105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 471/925] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:43:03 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:43:04.313320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7016db-f381-4692-a523-2894aa443f0c None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:04.313844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7016db-f381-4692-a523-2894aa443f0c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:04.313947 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7016db-f381-4692-a523-2894aa443f0c None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:04.315607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 455/926] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:04.321404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34886fb-6d8d-4538-887e-aea7db64d086 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:04.321693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34886fb-6d8d-4538-887e-aea7db64d086 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:04.321974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34886fb-6d8d-4538-887e-aea7db64d086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:04.340560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b34886fb-6d8d-4538-887e-aea7db64d086 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:04.341763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b34886fb-6d8d-4538-887e-aea7db64d086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:04.423907 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 472/927] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:04.446582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7016db-f381-4692-a523-2894aa443f0c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:04.556979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7016db-f381-4692-a523-2894aa443f0c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:04.591351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b1e9d-4d33-404b-a1e0-bb417d372caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:04.591919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b1e9d-4d33-404b-a1e0-bb417d372caf None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:04.592232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b1e9d-4d33-404b-a1e0-bb417d372caf None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:04.592511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b1e9d-4d33-404b-a1e0-bb417d372caf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:04.604098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b1e9d-4d33-404b-a1e0-bb417d372caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-ea9b1e9d-4d33-404b-a1e0-bb417d372caf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:04.608815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:04.610345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:04.611770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 456/928] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 10 16:43:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:04.616380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34886fb-6d8d-4538-887e-aea7db64d086 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:04.652542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:04.726814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34886fb-6d8d-4538-887e-aea7db64d086 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:04.762707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:04.763901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:04.764283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:04.764566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:04.765309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:04.765652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:04.767211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:04.775574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27612ed-e847-4d35-9550-087222fcf674 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:04.776063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27612ed-e847-4d35-9550-087222fcf674 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:04.776218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 473/929] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 10 16:43:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:04.776637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27612ed-e847-4d35-9550-087222fcf674 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:04.778400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 457/930] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:04.784428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:04.785041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5457d-99c1-408a-8940-be556c719c44 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:04.785627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5457d-99c1-408a-8940-be556c719c44 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:04.786102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5457d-99c1-408a-8940-be556c719c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:04.806071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a8f5457d-99c1-408a-8940-be556c719c44 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:04.807908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a8f5457d-99c1-408a-8940-be556c719c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:04.873331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:04.891006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:04.903960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dbb27-5c8b-4bd4-899f-2fab2599f741 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:04.908227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 458/931] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:04.930062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5457d-99c1-408a-8940-be556c719c44 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:04.950438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:04.951345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:04.951818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:04.952284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:04.966243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:04.967921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:04.983707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:04.984870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:04.985763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:04.985763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:04.985763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:04.992223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 474/932] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:04.993523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.051137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5457d-99c1-408a-8940-be556c719c44 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.088881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.089671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.090140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.090514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.091761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.093097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.096437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING keystone.server.flask.common [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 16:43:05.117500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.sql.core [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 16:43:05.117500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 16:43:05.117500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [parameters: {'id': '347b18b4cde64fd39e067a57fd565d9e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 16:43:05.117500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81540) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 16:43:05.130855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481508d-d0f1-4d3d-996a-26958bdfb527 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 347b18b4cde64fd39e067a57fd565d9e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 347b18b4cde64fd39e067a57fd565d9e. Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:43:05.134397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application result.read() Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 16:43:05.135223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application yield resource Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise self.value Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application session.commit() Aug 10 16:43:05.136222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self.session.flush() Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application compat.raise_( Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 16:43:05.137094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 16:43:05.138616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application result.read() Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:43:05.139465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [parameters: {'id': '347b18b4cde64fd39e067a57fd565d9e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.142409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:05.143525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 347b18b4cde64fd39e067a57fd565d9e. Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 459/933] 10.222.0.25 () {60 vars in 1140 bytes} [Wed Aug 10 16:43:04 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:05.144674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.182350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:05.192056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02888c-8c39-48c5-aeeb-b8961c0fa17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.192925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02888c-8c39-48c5-aeeb-b8961c0fa17b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.193352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02888c-8c39-48c5-aeeb-b8961c0fa17b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.193795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02888c-8c39-48c5-aeeb-b8961c0fa17b None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.194652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02888c-8c39-48c5-aeeb-b8961c0fa17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:05.195099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02888c-8c39-48c5-aeeb-b8961c0fa17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.197210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02888c-8c39-48c5-aeeb-b8961c0fa17b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.214933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 475/934] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:05.221693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1894-df47-45ac-ab73-a2f05cbc9bcc None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.221997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1894-df47-45ac-ab73-a2f05cbc9bcc None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.222373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1894-df47-45ac-ab73-a2f05cbc9bcc None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.223591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 476/935] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:05.225602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e1894-df47-45ac-ab73-a2f05cbc9bcc None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.229137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:05.267433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dda5f-1deb-476e-aa5e-4d16abf8dd43 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.306105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07adc4-2c4e-4fa9-bd75-9916ff6ebdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.307279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07adc4-2c4e-4fa9-bd75-9916ff6ebdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.307279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07adc4-2c4e-4fa9-bd75-9916ff6ebdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.307535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07adc4-2c4e-4fa9-bd75-9916ff6ebdf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.316037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07adc4-2c4e-4fa9-bd75-9916ff6ebdf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.318210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07adc4-2c4e-4fa9-bd75-9916ff6ebdf6 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-da07adc4-2c4e-4fa9-bd75-9916ff6ebdf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:43:05.330964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:05.332228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:05.333446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.339364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 460/936] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:05.339364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1893b-b3fa-4187-a689-052d5dc0168a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.339364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1893b-b3fa-4187-a689-052d5dc0168a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.339364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1893b-b3fa-4187-a689-052d5dc0168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.355419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8ed1893b-b3fa-4187-a689-052d5dc0168a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:05.357001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8ed1893b-b3fa-4187-a689-052d5dc0168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:05.366971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e1894-df47-45ac-ab73-a2f05cbc9bcc None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.404936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.405489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.405768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.406073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.417972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.419445 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:43:05.433207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:05.440038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 477/937] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:43:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:05.441428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.447218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 461/938] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:05.448128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1893b-b3fa-4187-a689-052d5dc0168a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.481020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:05.513838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:05.559693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155f190-ea29-4d21-9092-f41cc1c4c360 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.575694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1893b-b3fa-4187-a689-052d5dc0168a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.613646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.614604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.615076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.615544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.616669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:05.617188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.619591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.635434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.636655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.637130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.637578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.638818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:05.639322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.642490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.656594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 478/939] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:43:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:05.656594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.675826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 462/940] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 10 16:43:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:05.682017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.777296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9163a29-d5bd-4966-834c-013230034039 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.797890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:05.805762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:05.819666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ed008-e3a3-4e6e-a6fc-f07c536b09b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.820437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ed008-e3a3-4e6e-a6fc-f07c536b09b3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.820789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ed008-e3a3-4e6e-a6fc-f07c536b09b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.821136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ed008-e3a3-4e6e-a6fc-f07c536b09b3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.821727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ed008-e3a3-4e6e-a6fc-f07c536b09b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.823096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ed008-e3a3-4e6e-a6fc-f07c536b09b3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.827649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84cadc-2f87-4832-8409-c42b61f0c157 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-ed1ed008-e3a3-4e6e-a6fc-f07c536b09b3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:05.834179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:05.835423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:05.836718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 479/941] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Aug 10 16:43:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:05.837790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b02202-3a14-4d06-a815-3a33f078fa2d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.837790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b02202-3a14-4d06-a815-3a33f078fa2d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.837790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b02202-3a14-4d06-a815-3a33f078fa2d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.838781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 480/942] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:05.842842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b02202-3a14-4d06-a815-3a33f078fa2d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:05.883463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bb6e9-c7c5-4ea8-b37f-f4d76fc66d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:05.884111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb6e9-c7c5-4ea8-b37f-f4d76fc66d45 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.884441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb6e9-c7c5-4ea8-b37f-f4d76fc66d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.884769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb6e9-c7c5-4ea8-b37f-f4d76fc66d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.896057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bb6e9-c7c5-4ea8-b37f-f4d76fc66d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:05.898391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bb6e9-c7c5-4ea8-b37f-f4d76fc66d45 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-8f1bb6e9-c7c5-4ea8-b37f-f4d76fc66d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:05.916094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:05.917137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:05.918149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:05.918149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:05.918149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:05.918149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 463/943] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:05.919055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e79677-4ac1-4276-bd31-23025ac29577 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:05.919398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e79677-4ac1-4276-bd31-23025ac29577 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:05.919717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e79677-4ac1-4276-bd31-23025ac29577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:05.945980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-64e79677-4ac1-4276-bd31-23025ac29577 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:05.947404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-64e79677-4ac1-4276-bd31-23025ac29577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:05.963386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b02202-3a14-4d06-a815-3a33f078fa2d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.006348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.006964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.007248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.007632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.008155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:06.008474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.009847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.015877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 481/944] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:06.016423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.034244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 464/945] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:06.040432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e79677-4ac1-4276-bd31-23025ac29577 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.088982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:06.114178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:06.154454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44227db7-55fa-4c43-863f-6f022f2a9b78 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.154669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e79677-4ac1-4276-bd31-23025ac29577 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.196893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.197561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.197984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.198340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.199016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:06.199391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.200295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.200953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.201106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.201459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.201832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.202561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:06.202963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.204447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.210198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 465/946] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:06.210879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.225493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 482/947] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:06.225493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.343293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d72128-b62e-4fda-804d-9ba96e0bb295 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.347434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3d1c2-e7f0-48d6-a054-70cfbb187ed2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.355715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:06.378670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:06.381920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.382555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.382835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.383104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.383770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.385243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.389247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.390228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.391082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.391356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.392027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.393685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:06.404313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:06.407024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:06.409090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 466/948] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:06.410889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.428203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 483/949] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:43:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:43:06.428203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.565038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49ddb-9f69-4132-ade3-94686522901b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.588754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d631-6726-439f-ab2b-c091a299494e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.615462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.616744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.617020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.617333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.628914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.630478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.643738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.644386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.644665 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.644936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.645572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:06.645871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.647335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:43:06.647619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:06.648513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:06.649960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 467/950] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:06.657488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.658625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 484/951] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:06.664205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.702191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:06.719533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:06.781483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9f987-09a7-477c-8f70-5a6362ea039c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.802786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db327e55-b1df-4a69-b077-39aaf1032827 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:06.827454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.828837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.829123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.829402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.830581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:06.830904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.832839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.838399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:06.839147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:06.839527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:06.839900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:06.840710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:06.841113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:06.842614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:06.851889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 468/952] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:43:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:06.859452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 485/953] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:06.860511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:06.867148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:07.005576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:07.018262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63db62c-2cc8-4f6c-954e-482492d86890 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:07.029457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8571803-b7d2-4794-be56-5009974a68c8 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:07.056667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:07.056910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc6d8a-3a01-4ccb-86ce-248a9deb3598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:07.057394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc6d8a-3a01-4ccb-86ce-248a9deb3598 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:07.057674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc6d8a-3a01-4ccb-86ce-248a9deb3598 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:07.057950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc6d8a-3a01-4ccb-86ce-248a9deb3598 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:07.058755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc6d8a-3a01-4ccb-86ce-248a9deb3598 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:07.060019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc6d8a-3a01-4ccb-86ce-248a9deb3598 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:07.095683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7912e6-d815-406a-a22d-157feb147373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:07.096907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7912e6-d815-406a-a22d-157feb147373 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:07.097403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7912e6-d815-406a-a22d-157feb147373 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:07.097912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7912e6-d815-406a-a22d-157feb147373 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/81fba04097094820be201ad24ed2658d/roles/457c69fdf6cc4e33885c6cc4bac6b9ec` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:07.100214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 486/954] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:43:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:07.109930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7912e6-d815-406a-a22d-157feb147373 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=81fba04097094820be201ad24ed2658d, role_id=457c69fdf6cc4e33885c6cc4bac6b9ec)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:07.111814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7912e6-d815-406a-a22d-157feb147373 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:07.130295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 469/955] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:43:06 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/81fba04097094820be201ad24ed2658d/roles/457c69fdf6cc4e33885c6cc4bac6b9ec => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:43:07.919381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228849a-ec46-4dc7-84ed-03286cde8c31 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:07.919931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228849a-ec46-4dc7-84ed-03286cde8c31 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:07.920012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228849a-ec46-4dc7-84ed-03286cde8c31 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:07.921869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 487/956] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:07.927622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0c9d1-47f5-47e4-90e2-aece6fced17a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:07.928092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0c9d1-47f5-47e4-90e2-aece6fced17a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:07.928200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0c9d1-47f5-47e4-90e2-aece6fced17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:07.946960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f6d0c9d1-47f5-47e4-90e2-aece6fced17a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:07.948181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f6d0c9d1-47f5-47e4-90e2-aece6fced17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:08.029294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 470/957] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:08.056619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228849a-ec46-4dc7-84ed-03286cde8c31 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:08.197704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228849a-ec46-4dc7-84ed-03286cde8c31 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:08.251808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:08.252733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:08.253163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:08.253606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:08.254575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:08.256912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:08.260815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING keystone.server.flask.common [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 16:43:08.280268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.sql.core [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 16:43:08.280268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 16:43:08.280268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [parameters: {'user_id': '57ac60086a5a4f3482c3ddc8d8db433b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 16:43:08.280268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81539) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result.read() Aug 10 16:43:08.292954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:43:08.293863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application yield resource Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise self.value Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application session.commit() Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 16:43:08.294814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.session.flush() Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application compat.raise_( Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 16:43:08.295711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:43:08.298992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application result.read() Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:43:08.305173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [parameters: {'user_id': '57ac60086a5a4f3482c3ddc8d8db433b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:08.307251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 16:43:08.309352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 16:43:08.311508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 16:43:08.311508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 16:43:08.311508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 16:43:08.311508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 16:43:08.311508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 16:43:08.311508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:08.311508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 488/958] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 10 16:43:08 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:08.314685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0c9d1-47f5-47e4-90e2-aece6fced17a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:08.445020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:08.452885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0c9d1-47f5-47e4-90e2-aece6fced17a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:08.491174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a721d90-a5ac-4a1b-ac38-81464980f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:08.491826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721d90-a5ac-4a1b-ac38-81464980f06e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:08.492251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721d90-a5ac-4a1b-ac38-81464980f06e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:08.492529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721d90-a5ac-4a1b-ac38-81464980f06e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:08.500603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a721d90-a5ac-4a1b-ac38-81464980f06e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:08.502270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a721d90-a5ac-4a1b-ac38-81464980f06e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-2a721d90-a5ac-4a1b-ac38-81464980f06e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:43:08.515152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:08.516196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:08.517163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 471/959] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:43:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:08.525854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:08.539339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:08.583135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d731bc7-80fe-465c-b7e1-a15ef72d57da None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:08.583453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d731bc7-80fe-465c-b7e1-a15ef72d57da None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:08.583702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d731bc7-80fe-465c-b7e1-a15ef72d57da None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:08.585495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 472/960] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:08.591190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0ca3-1e1e-4f84-b1ae-0f0e4b57c469 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:08.591721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0ca3-1e1e-4f84-b1ae-0f0e4b57c469 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:08.592249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0ca3-1e1e-4f84-b1ae-0f0e4b57c469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:08.612097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8ffd0ca3-1e1e-4f84-b1ae-0f0e4b57c469 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:08.613660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8ffd0ca3-1e1e-4f84-b1ae-0f0e4b57c469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:08.648078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3f46-c0dd-4ad0-a858-ae07ea3dd0f5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:08.692189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:08.693206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:08.693793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:08.694287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:08.702753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 473/961] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:08.703454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:08.703454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:08.703454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:08.718141 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd0ca3-1e1e-4f84-b1ae-0f0e4b57c469 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:08.762128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 489/962] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 10 16:43:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:08.824025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:08.842628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd0ca3-1e1e-4f84-b1ae-0f0e4b57c469 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:08.886766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:08.887427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:08.887706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:08.888050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:08.888681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:08.890228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:08.900341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:08.901934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:08.903334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 474/963] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:08.907826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:08.993790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:09.032471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba35fc4-2900-4171-b2ac-baa92f007c3b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:09.071122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cb0f9-23f7-4c04-a57a-9b9ba6603710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:09.071748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cb0f9-23f7-4c04-a57a-9b9ba6603710 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.072085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cb0f9-23f7-4c04-a57a-9b9ba6603710 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.072391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cb0f9-23f7-4c04-a57a-9b9ba6603710 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.073031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cb0f9-23f7-4c04-a57a-9b9ba6603710 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:09.073333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cb0f9-23f7-4c04-a57a-9b9ba6603710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:09.074706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cb0f9-23f7-4c04-a57a-9b9ba6603710 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:09.080684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 490/964] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:43:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:09.086386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:09.197225 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83718101-7aa8-4b9e-ac2f-dccbc98b5331 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:09.231739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeacb21-038f-478a-9b1e-67cb55cdc018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:09.232390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeacb21-038f-478a-9b1e-67cb55cdc018 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.232671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeacb21-038f-478a-9b1e-67cb55cdc018 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.232945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeacb21-038f-478a-9b1e-67cb55cdc018 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.233577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeacb21-038f-478a-9b1e-67cb55cdc018 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:09.233876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeacb21-038f-478a-9b1e-67cb55cdc018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:09.235275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeacb21-038f-478a-9b1e-67cb55cdc018 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:09.245775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 475/965] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:43:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:09.452729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa494f-679d-4c3f-959c-153166df37e8 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.453202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa494f-679d-4c3f-959c-153166df37e8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.453620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa494f-679d-4c3f-959c-153166df37e8 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.455861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 491/966] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:09.463612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646454b0-020e-44c2-a3c5-4a456834422f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.464204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646454b0-020e-44c2-a3c5-4a456834422f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.464652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646454b0-020e-44c2-a3c5-4a456834422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.491869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-646454b0-020e-44c2-a3c5-4a456834422f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:09.493894 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-646454b0-020e-44c2-a3c5-4a456834422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:09.505994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:09.573388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 476/967] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:09.596055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa494f-679d-4c3f-959c-153166df37e8 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:09.703097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa494f-679d-4c3f-959c-153166df37e8 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:09.732088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:09.739210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:09.739852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.740133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.740428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.751665 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:09.756522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:09.757442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:09.758334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:09.759695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 492/968] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 10 16:43:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:09.763654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646454b0-020e-44c2-a3c5-4a456834422f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:09.878679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646454b0-020e-44c2-a3c5-4a456834422f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:09.917311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f0d74-d36d-410b-8fa0-8b1cdd07d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:09.918114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f0d74-d36d-410b-8fa0-8b1cdd07d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.918534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f0d74-d36d-410b-8fa0-8b1cdd07d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.918946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f0d74-d36d-410b-8fa0-8b1cdd07d0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.919738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f0d74-d36d-410b-8fa0-8b1cdd07d0d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:09.920192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f0d74-d36d-410b-8fa0-8b1cdd07d0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:09.922351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f0d74-d36d-410b-8fa0-8b1cdd07d0d8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:09.936206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 477/969] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 10 16:43:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:09.938074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:09.976744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce165e-be04-4f35-bfff-2569b925b34d None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.977004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce165e-be04-4f35-bfff-2569b925b34d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.977357 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce165e-be04-4f35-bfff-2569b925b34d None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.978824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 478/970] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:09.983636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7908ba-cfd3-4106-a273-9a4d2bd1f576 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:09.983938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7908ba-cfd3-4106-a273-9a4d2bd1f576 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:09.984828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7908ba-cfd3-4106-a273-9a4d2bd1f576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:09.998269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:10.005860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7e7908ba-cfd3-4106-a273-9a4d2bd1f576 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:10.008800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7e7908ba-cfd3-4106-a273-9a4d2bd1f576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:10.081499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b910500-cd0c-43ae-8499-353921d75ebb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.120188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 479/971] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:10.133721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.134481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.134892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.135279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.139650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7908ba-cfd3-4106-a273-9a4d2bd1f576 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.149976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.151852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:10.169120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:10.170129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:10.171788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:10.171788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:10.171788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:10.171788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 493/972] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:10.177259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.231618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:10.282793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7908ba-cfd3-4106-a273-9a4d2bd1f576 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.299336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cdd3-fe6c-4d82-8e3e-a6f34702b8f1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.328270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.328270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.328270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.328270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.328694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.328694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:10.339543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:10.346013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:10.346828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:10.347633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 480/973] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:10.347633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.359178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.360338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.360631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.360909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.361600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:10.361891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.363354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:10.380201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 494/974] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:10.383125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.506461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe4d74-b83a-461c-962a-121b91e6958e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.519777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104bb43-4553-4dc7-ae64-e8a87b03e5e5 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.521530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:10.568980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.569740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.570226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.570666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.571169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.571693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:10.572154 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.572263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.572624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.573066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.574573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:10.588169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 481/975] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:10.588330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.588330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:10.589821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:43:10.607111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:10.608065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:43:10.609056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:10.612759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 495/976] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:10.617027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.674105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:10.733233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692c24f-b71e-428d-b0e9-74a1e79a7a0a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.750214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72621fd-18a3-42e5-9937-9bda48361a50 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.780074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.781344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.781814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.782231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.783009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:10.783488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.785045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.785230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:10.785758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.786034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.786298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.787019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:10.787319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.788777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:10.793314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 482/977] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:10.800487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.807158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 496/978] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:10.813931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:10.847898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:10.948407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4df1-a855-4f00-82d3-0e639456b4fb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.953862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d8e88-2b85-43a1-92a0-942d881d7d89 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:10.977521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:10.991588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:10.993147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:10.993911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:10.994343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:10.996035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] RBAC: Adding query filter params (interface=admin, service_id=08dabb19ccc24c7bbf2b3dc06f951b93, region_id=RegionOne) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:10.996860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:10.999116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.000889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:11.001500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.001785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.002056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.009655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 483/979] 10.222.0.25 () {56 vars in 1249 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/endpoints?service_id=08dabb19ccc24c7bbf2b3dc06f951b93&interface=admin®ion_id=RegionOne => generated 193 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:11.016070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:11.075297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:11.076826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.099742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:11.158335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147e5b0-c488-44a3-88a7-6928d70f6b3c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:11.188744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 497/980] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 16:43:11.194803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:11.203424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbe9b8-3f3c-4f6a-8ff0-035fd5eac36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:11.204060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbe9b8-3f3c-4f6a-8ff0-035fd5eac36a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.204352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbe9b8-3f3c-4f6a-8ff0-035fd5eac36a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.204623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbe9b8-3f3c-4f6a-8ff0-035fd5eac36a None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.205224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbe9b8-3f3c-4f6a-8ff0-035fd5eac36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:11.206587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbe9b8-3f3c-4f6a-8ff0-035fd5eac36a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.213992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 484/981] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:43:11 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:43:11.285831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3897ed-8ec7-4ad8-b4d7-4ab9f08dcadf None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.286118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3897ed-8ec7-4ad8-b4d7-4ab9f08dcadf None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.286395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3897ed-8ec7-4ad8-b4d7-4ab9f08dcadf None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.287957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 485/982] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:11.293444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803425-0a6e-4bd2-8ea6-8b2eb78ced89 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.293728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803425-0a6e-4bd2-8ea6-8b2eb78ced89 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.294011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803425-0a6e-4bd2-8ea6-8b2eb78ced89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.313761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f6803425-0a6e-4bd2-8ea6-8b2eb78ced89 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:11.315002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f6803425-0a6e-4bd2-8ea6-8b2eb78ced89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:11.315115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1851-d6e7-474e-adb0-a1f2a26943f7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:11.344088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:11.360518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:11.361126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.361466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.361803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.373336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:11.374839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.378863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:11.393062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:11.394211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:11.395269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:11.395269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:11.395269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:11.395269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 498/983] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:11.399341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 486/984] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:11.401246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:11.425255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6803425-0a6e-4bd2-8ea6-8b2eb78ced89 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:11.513806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ac108-c329-40d6-b509-f0e008f1143e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:11.529889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6803425-0a6e-4bd2-8ea6-8b2eb78ced89 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:11.550696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:11.551312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.551597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.551873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.552558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:11.552875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:11.554243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.567253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 499/985] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:11.568637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:11.569255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.569532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.569803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.570463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:11.571830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.572619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:11.590466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 487/986] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:43:11 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:11.616243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:11.695953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea0329-c17f-45c3-9de7-fac9d99a0a53 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:11.742240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:11.742880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.743155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.743429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.751284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:11.752763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:43:11.764254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:11.765149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:11.766069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 500/987] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:11.773282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:11.824987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:11.881856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c723a-1c2d-47a5-ae8e-ce899d73cee7 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:11.915321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba670a-a89a-442f-9e54-9ff61d1079d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:11.915935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba670a-a89a-442f-9e54-9ff61d1079d5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:11.916237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba670a-a89a-442f-9e54-9ff61d1079d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:11.916510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba670a-a89a-442f-9e54-9ff61d1079d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:11.917196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba670a-a89a-442f-9e54-9ff61d1079d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:11.917496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba670a-a89a-442f-9e54-9ff61d1079d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:11.918918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba670a-a89a-442f-9e54-9ff61d1079d5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:11.937048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 488/988] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:11.939181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:12.032939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:12.047251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b05d46-68d9-4f1c-b221-d3cefa097055 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:12.081502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ef57a-a6ed-4400-88e8-38741bb92e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:12.082116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ef57a-a6ed-4400-88e8-38741bb92e54 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:12.082396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ef57a-a6ed-4400-88e8-38741bb92e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:12.082669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ef57a-a6ed-4400-88e8-38741bb92e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:12.083435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ef57a-a6ed-4400-88e8-38741bb92e54 None admin] RBAC: Adding query filter params (role.id=457c69fdf6cc4e33885c6cc4bac6b9ec, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=81fba04097094820be201ad24ed2658d) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:12.083744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ef57a-a6ed-4400-88e8-38741bb92e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:12.085200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ef57a-a6ed-4400-88e8-38741bb92e54 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:12.096169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 501/989] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:11 2022] GET /identity/v3/role_assignments?user.id=81fba04097094820be201ad24ed2658d&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=457c69fdf6cc4e33885c6cc4bac6b9ec => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:13.257273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba4c78-c530-49bb-8223-016a37362f68 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:13.258007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba4c78-c530-49bb-8223-016a37362f68 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:13.258007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba4c78-c530-49bb-8223-016a37362f68 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:13.259361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 489/990] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:13.265915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec2bb4-b384-4363-86d4-9912e838dcff None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:13.266595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec2bb4-b384-4363-86d4-9912e838dcff None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:13.266829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec2bb4-b384-4363-86d4-9912e838dcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:13.294195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8fec2bb4-b384-4363-86d4-9912e838dcff None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:13.296567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8fec2bb4-b384-4363-86d4-9912e838dcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:13.400218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 502/991] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:13.422407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba4c78-c530-49bb-8223-016a37362f68 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:13.580336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba4c78-c530-49bb-8223-016a37362f68 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:13.591906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:13.607832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65aba24-01a6-4e3c-a4a0-48e8a977d13e None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:13.608310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65aba24-01a6-4e3c-a4a0-48e8a977d13e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:13.608748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65aba24-01a6-4e3c-a4a0-48e8a977d13e None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:13.611092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 503/992] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:13.619519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573698e-a7dd-498a-acbd-4101d6d60f93 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:13.620049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573698e-a7dd-498a-acbd-4101d6d60f93 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:13.620529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573698e-a7dd-498a-acbd-4101d6d60f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:13.628128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:13.628785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:13.629060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:13.629325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:13.629903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:13.631260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:13.641348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:13.642337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:43:13.643199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:13.644865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 490/993] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 10 16:43:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:13.649370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:13.654146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5573698e-a7dd-498a-acbd-4101d6d60f93 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:13.655440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5573698e-a7dd-498a-acbd-4101d6d60f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:13.741282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 504/994] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:13.764048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573698e-a7dd-498a-acbd-4101d6d60f93 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:13.764910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc70f3-6eb2-4784-a795-b873caa7edb5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:13.779613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:13.815503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:13.816730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:13.817122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:13.817537 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:13.818580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:13.819075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:13.821264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:13.829754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 491/995] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:13.835222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:13.877625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:13.891060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573698e-a7dd-498a-acbd-4101d6d60f93 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:13.940375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:13.941534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:13.941823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:13.942114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:13.943063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:13.944464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:13.954077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:13.955266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:13.957814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 505/996] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:13.961084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:13.963548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd2400-7ede-4864-b81d-86d0dc32487e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.007998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.008641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.008916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.009185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.009812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:14.010109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.011480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.017607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 492/997] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 10 16:43:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:14.023522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:14.058234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:14.091274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a7cf2-e895-4200-9b7f-ecdc066f2b87 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.137082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6315744-62d4-452e-bfb7-d85b0f96f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.137697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6315744-62d4-452e-bfb7-d85b0f96f483 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.137974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6315744-62d4-452e-bfb7-d85b0f96f483 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.138237 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6315744-62d4-452e-bfb7-d85b0f96f483 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.138862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6315744-62d4-452e-bfb7-d85b0f96f483 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:14.139159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6315744-62d4-452e-bfb7-d85b0f96f483 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.140806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6315744-62d4-452e-bfb7-d85b0f96f483 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.144033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:14.145593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 506/998] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:43:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:14.146946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24765d1-c544-478a-99fe-800137eb5df1 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.147315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24765d1-c544-478a-99fe-800137eb5df1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.147631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24765d1-c544-478a-99fe-800137eb5df1 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.149184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3467e57-30e0-4c7d-844a-b19b2de3780e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.149357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 507/999] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:14.151322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24765d1-c544-478a-99fe-800137eb5df1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:14.192281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a57a-1c02-437c-89d2-7f99ab10d0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.192900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a57a-1c02-437c-89d2-7f99ab10d0f1 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.193279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a57a-1c02-437c-89d2-7f99ab10d0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.193558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a57a-1c02-437c-89d2-7f99ab10d0f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.194245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a57a-1c02-437c-89d2-7f99ab10d0f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.195729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a57a-1c02-437c-89d2-7f99ab10d0f1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.214394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 493/1000] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:43:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:14.215697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39766-47ed-4d91-b8cd-3d5fce5bbbcd None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.215990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39766-47ed-4d91-b8cd-3d5fce5bbbcd None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.216359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39766-47ed-4d91-b8cd-3d5fce5bbbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.240841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-13a39766-47ed-4d91-b8cd-3d5fce5bbbcd None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:14.242765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-13a39766-47ed-4d91-b8cd-3d5fce5bbbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:14.310160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24765d1-c544-478a-99fe-800137eb5df1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.340412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 494/1001] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:14.358962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.360188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.360511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.360871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.361553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:14.361911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.363557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.366995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:14.369877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39766-47ed-4d91-b8cd-3d5fce5bbbcd None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:14.370745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 508/1002] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:43:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:14.377381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:14.488572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:14.506521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39766-47ed-4d91-b8cd-3d5fce5bbbcd None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.526137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07dd0f-27bf-4b38-8465-c7c236bf7290 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.564925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.565758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.566528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.566937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] PATH_INFO: `/v3/services/image` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.567924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.569895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.572782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.573399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.573677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.573947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.574652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] RBAC: Adding query filter params (interface=public, service_id=e435ba863e9d4529bc10ae1dbfb2ca9c, region_id=RegionOne) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:14.574960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.576338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:14.583043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:14.584310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:14.585915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:14.587312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 509/1003] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:43:14 2022] GET /identity/v3/endpoints?service_id=e435ba863e9d4529bc10ae1dbfb2ca9c&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:14.595099 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 495/1004] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Aug 10 16:43:14 2022] GET /identity/v3/services/image => generated 86 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:14.595293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:14.598966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:14.680255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:14.744514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d394-fb99-4a2a-b19e-92b65f806b95 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.749194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a97c94-708d-4823-a87a-328d3bdc2125 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:14.798580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.799485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.799916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.800407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.801610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.802590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0ce44-cf28-4d1a-8336-9112f2adcbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:14.803504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0ce44-cf28-4d1a-8336-9112f2adcbfb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:14.803932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.804047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0ce44-cf28-4d1a-8336-9112f2adcbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:14.804408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0ce44-cf28-4d1a-8336-9112f2adcbfb None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:14.805438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0ce44-cf28-4d1a-8336-9112f2adcbfb None admin] RBAC: Adding query filter params (name=image) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:14.805931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0ce44-cf28-4d1a-8336-9112f2adcbfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:14.808122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0ce44-cf28-4d1a-8336-9112f2adcbfb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:14.820496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 510/1005] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:43:14 2022] GET /identity/v3/services => generated 1727 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:43:14.820715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 496/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:14.828298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:15.005195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02a97-6b97-478f-8f8a-5ec776ea1369 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:15.050269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:15.063438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef7b26-6235-482b-8113-975abf5b1cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:15.064530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef7b26-6235-482b-8113-975abf5b1cfd None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:15.065017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef7b26-6235-482b-8113-975abf5b1cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:15.065485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef7b26-6235-482b-8113-975abf5b1cfd None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:15.066873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef7b26-6235-482b-8113-975abf5b1cfd None admin] RBAC: Adding query filter params (type=image) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:15.067410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef7b26-6235-482b-8113-975abf5b1cfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:15.069883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef7b26-6235-482b-8113-975abf5b1cfd None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:15.078600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 511/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:16.373408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa896d-1518-413f-9c03-ba27757eed04 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:16.373947 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa896d-1518-413f-9c03-ba27757eed04 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:16.374260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa896d-1518-413f-9c03-ba27757eed04 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:16.376618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 497/1008] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:16.382138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ee91-4e2f-4a82-ad66-5c2740c73422 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:16.382745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ee91-4e2f-4a82-ad66-5c2740c73422 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:16.382745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ee91-4e2f-4a82-ad66-5c2740c73422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:16.401400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d0c0ee91-4e2f-4a82-ad66-5c2740c73422 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:16.404200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d0c0ee91-4e2f-4a82-ad66-5c2740c73422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:16.481592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 512/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:16.504710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa896d-1518-413f-9c03-ba27757eed04 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:16.524256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:16.611982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa896d-1518-413f-9c03-ba27757eed04 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:16.647066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa33a5-6401-4681-b5d5-df7e87402f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:16.647616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa33a5-6401-4681-b5d5-df7e87402f12 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:16.647892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa33a5-6401-4681-b5d5-df7e87402f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:16.648191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa33a5-6401-4681-b5d5-df7e87402f12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:16.649294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa33a5-6401-4681-b5d5-df7e87402f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:16.650721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa33a5-6401-4681-b5d5-df7e87402f12 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:16.652504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 498/1010] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:16.658133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0ee91-4e2f-4a82-ad66-5c2740c73422 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:16.784249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0ee91-4e2f-4a82-ad66-5c2740c73422 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:16.807287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:16.821625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a5ad7-bfbe-4e2b-adc6-2e600a288522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:16.822241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a5ad7-bfbe-4e2b-adc6-2e600a288522 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:16.822517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a5ad7-bfbe-4e2b-adc6-2e600a288522 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:16.822793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a5ad7-bfbe-4e2b-adc6-2e600a288522 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:16.823444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a5ad7-bfbe-4e2b-adc6-2e600a288522 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:16.824904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a5ad7-bfbe-4e2b-adc6-2e600a288522 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:16.855364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 513/1011] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:43:16 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:43:17.189298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f80d7-e51a-4554-b1e5-13ae05bfd0b2 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.189726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f80d7-e51a-4554-b1e5-13ae05bfd0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.190140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f80d7-e51a-4554-b1e5-13ae05bfd0b2 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.192849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 499/1012] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:17.200185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f2cfa-40b4-44e9-bd2d-c7b3ae9de0c3 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.200645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f2cfa-40b4-44e9-bd2d-c7b3ae9de0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.200990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f2cfa-40b4-44e9-bd2d-c7b3ae9de0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.221235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-736f2cfa-40b4-44e9-bd2d-c7b3ae9de0c3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:17.222627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-736f2cfa-40b4-44e9-bd2d-c7b3ae9de0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:17.257356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914932e8-c486-45ec-922e-79dcc0c17c01 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.258264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914932e8-c486-45ec-922e-79dcc0c17c01 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.258573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914932e8-c486-45ec-922e-79dcc0c17c01 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.261013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 500/1013] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:17.269635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef619e0-3a30-4427-a0f2-112d00a8797f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.270480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef619e0-3a30-4427-a0f2-112d00a8797f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.271192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef619e0-3a30-4427-a0f2-112d00a8797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.299561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8ef619e0-3a30-4427-a0f2-112d00a8797f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:17.301512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8ef619e0-3a30-4427-a0f2-112d00a8797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:17.317068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 514/1014] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:17.343081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f2cfa-40b4-44e9-bd2d-c7b3ae9de0c3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:17.402729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 501/1015] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:17.426183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef619e0-3a30-4427-a0f2-112d00a8797f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:17.442647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:17.474966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f2cfa-40b4-44e9-bd2d-c7b3ae9de0c3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:17.478289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:17.514581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:17.515337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.515578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.515853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.516471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:17.517852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:17.527250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:17.532512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:17.533651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 515/1016] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:17.534691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:17.548187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef619e0-3a30-4427-a0f2-112d00a8797f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:17.587564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:17.588214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.588500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.589027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.589418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:17.590797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:17.608513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 502/1017] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:43:17 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:17.687794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd508-4ddf-4290-925b-8f6b4e57c37d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:17.753377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:17.754212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.754635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.755048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.756010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:17.756507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:17.758718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:17.768337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 516/1018] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:43:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:17.773904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:17.854760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:17.884721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82278b96-7166-48cb-9265-60962e05c1ce None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:17.920760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b82e2-ee85-4aac-809a-04cd184d9233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:17.921405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b82e2-ee85-4aac-809a-04cd184d9233 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:17.921685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b82e2-ee85-4aac-809a-04cd184d9233 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:17.921959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b82e2-ee85-4aac-809a-04cd184d9233 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:17.922588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b82e2-ee85-4aac-809a-04cd184d9233 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:17.922956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b82e2-ee85-4aac-809a-04cd184d9233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:17.924375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b82e2-ee85-4aac-809a-04cd184d9233 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:17.934393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 503/1019] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 10 16:43:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:17.936091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:17.968926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:18.049309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e2d65-5a18-4d28-b40f-5d15abea46c1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:18.085365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a211c0-7b31-4094-9e4b-75dccd6e919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:18.085987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a211c0-7b31-4094-9e4b-75dccd6e919f None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:18.086282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a211c0-7b31-4094-9e4b-75dccd6e919f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:18.086547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a211c0-7b31-4094-9e4b-75dccd6e919f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:18.087211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a211c0-7b31-4094-9e4b-75dccd6e919f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:18.088622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a211c0-7b31-4094-9e4b-75dccd6e919f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:18.112180 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 517/1020] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:43:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:18.934814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b42f6-41c4-4d2a-92c1-81c74913cd30 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:18.935351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b42f6-41c4-4d2a-92c1-81c74913cd30 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:18.935351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b42f6-41c4-4d2a-92c1-81c74913cd30 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:18.936896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 504/1021] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:18.942330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad04eb-4065-4658-8ce6-dcc6a043db3f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:18.943042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad04eb-4065-4658-8ce6-dcc6a043db3f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:18.943042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad04eb-4065-4658-8ce6-dcc6a043db3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:18.961264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0cad04eb-4065-4658-8ce6-dcc6a043db3f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:18.962525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0cad04eb-4065-4658-8ce6-dcc6a043db3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:19.041481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 518/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:19.065551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b42f6-41c4-4d2a-92c1-81c74913cd30 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:19.170923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b42f6-41c4-4d2a-92c1-81c74913cd30 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:19.205076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11fd1a-7956-41e3-8b55-37b4a41ed8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:19.205623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11fd1a-7956-41e3-8b55-37b4a41ed8fd None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:19.205899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11fd1a-7956-41e3-8b55-37b4a41ed8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:19.206168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11fd1a-7956-41e3-8b55-37b4a41ed8fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:19.207262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11fd1a-7956-41e3-8b55-37b4a41ed8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:19.208722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11fd1a-7956-41e3-8b55-37b4a41ed8fd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:19.210497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 505/1023] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:19.216071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad04eb-4065-4658-8ce6-dcc6a043db3f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:19.228632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:19.318204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad04eb-4065-4658-8ce6-dcc6a043db3f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:19.356016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e99ba1-1b63-4e72-b243-b1a01145d7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:19.356660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99ba1-1b63-4e72-b243-b1a01145d7a2 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:19.356938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99ba1-1b63-4e72-b243-b1a01145d7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:19.357205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99ba1-1b63-4e72-b243-b1a01145d7a2 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:19.357851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e99ba1-1b63-4e72-b243-b1a01145d7a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:19.361289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e99ba1-1b63-4e72-b243-b1a01145d7a2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:19.383147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 519/1024] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:43:19 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:43:19.638323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334db1-bea9-4027-aed3-c5f99e659285 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:19.638510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334db1-bea9-4027-aed3-c5f99e659285 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:19.638788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334db1-bea9-4027-aed3-c5f99e659285 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:19.640722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 506/1025] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:19.645872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e7664-6a0d-46d5-b761-9f86cbe2df5d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:19.646161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e7664-6a0d-46d5-b761-9f86cbe2df5d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:19.646436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e7664-6a0d-46d5-b761-9f86cbe2df5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:19.664499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-fc2e7664-6a0d-46d5-b761-9f86cbe2df5d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:19.665678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fc2e7664-6a0d-46d5-b761-9f86cbe2df5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:19.726276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:19.735622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 520/1026] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:19.757902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a334db1-bea9-4027-aed3-c5f99e659285 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:19.767922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:19.866415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a334db1-bea9-4027-aed3-c5f99e659285 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:19.901402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:19.901954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:19.902228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:19.902494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:19.903069 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:19.904640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:19.913924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:19.914910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:19.915734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 507/1027] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Aug 10 16:43:19 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:19.923134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e7664-6a0d-46d5-b761-9f86cbe2df5d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:20.027087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e7664-6a0d-46d5-b761-9f86cbe2df5d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:20.062796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:20.063411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.063687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.063953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.064660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] RBAC: Adding query filter params (name=image) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:20.064963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:20.066585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:20.076170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 521/1028] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:20.077393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:20.150601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:20.183849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7a428-d9eb-4c42-8aed-774ef0a835c8 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:20.220085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04951025-ba04-4a79-b216-f8ff4d90ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:20.220728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04951025-ba04-4a79-b216-f8ff4d90ce56 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.221002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04951025-ba04-4a79-b216-f8ff4d90ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.221271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04951025-ba04-4a79-b216-f8ff4d90ce56 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.221904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04951025-ba04-4a79-b216-f8ff4d90ce56 None admin] RBAC: Adding query filter params (type=image) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:20.222227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04951025-ba04-4a79-b216-f8ff4d90ce56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:20.223600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04951025-ba04-4a79-b216-f8ff4d90ce56 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:20.229150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 508/1029] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:20.235320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:20.253133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aac6c-3c8e-479d-a30e-c588faee5189 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.253546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aac6c-3c8e-479d-a30e-c588faee5189 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.253692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aac6c-3c8e-479d-a30e-c588faee5189 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.258714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 509/1030] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:20.260334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f986076-a7ea-4f3a-8919-f918fa830725 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.260625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f986076-a7ea-4f3a-8919-f918fa830725 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.260928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f986076-a7ea-4f3a-8919-f918fa830725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.279456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5f986076-a7ea-4f3a-8919-f918fa830725 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:20.281217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5f986076-a7ea-4f3a-8919-f918fa830725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:20.302642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:20.347422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12826877-b2ed-4a18-9cee-c052eb4ef02b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:20.364169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 510/1031] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:20.379722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f986076-a7ea-4f3a-8919-f918fa830725 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:20.382927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:20.383544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.383821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.384088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.384817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] RBAC: Adding query filter params (interface=public, service_id=f29c853669de43cabe79c7319f4c9b4a, region_id=RegionOne) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:20.385127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:20.386482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:20.392042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 522/1032] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:43:20 2022] GET /identity/v3/endpoints?service_id=f29c853669de43cabe79c7319f4c9b4a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:20.397796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:20.444575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:20.492639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f986076-a7ea-4f3a-8919-f918fa830725 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:20.507955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa67d-cffe-4c07-b5ce-14a548128f48 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:20.532773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:20.537744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:20.538878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.539335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.540055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.544701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:20.545327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.545609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.545883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.547035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:20.548434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:20.554060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:20.555192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 523/1033] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:43:20 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:20.559202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:20.560365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:20.561364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:20.563826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 511/1034] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 10 16:43:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:20.570170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:20.688747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6425-db6f-4988-b85a-ced78c4c21f6 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:20.725849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:20.726391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.726674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.726946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.727649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:20.727952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:20.729368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:20.738112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 524/1035] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 10 16:43:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:20.743907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:20.854390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d96f7-1fca-4edf-8e37-297a85f05b69 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:20.878830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:20.891001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:20.891635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:20.891914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:20.892212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:20.903340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:20.904835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:20.920376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:20.921430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:20.922300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:20.922300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:20.922300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:20.922300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 512/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:20.929212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.002770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:21.057686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f0c95-feca-4fcf-9a91-9a52be04439f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:21.101263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:21.101892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.102176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.102563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.103747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:21.104296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:21.106698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:21.123254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 525/1037] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:21.125456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.230019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df6ac2-8ff0-4d1a-be0e-ac3c2c989268 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:21.265016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:21.265846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.266079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.266348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] PATH_INFO: `/v3/users/nova` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.274261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:21.275703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:43:21.287089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:21.288052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:43:21.289333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:21.292213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 513/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:43:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:21.294300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.411130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554bb3b-3c33-4ace-9ed9-6acda9d282d3 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:21.423797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:21.453656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e7f4f-05b8-47ee-9c5e-dd83eb5822e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:21.454273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e7f4f-05b8-47ee-9c5e-dd83eb5822e7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.454548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e7f4f-05b8-47ee-9c5e-dd83eb5822e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.455492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e7f4f-05b8-47ee-9c5e-dd83eb5822e7 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.455640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e7f4f-05b8-47ee-9c5e-dd83eb5822e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:21.455932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e7f4f-05b8-47ee-9c5e-dd83eb5822e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:21.457418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e7f4f-05b8-47ee-9c5e-dd83eb5822e7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:21.480707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 526/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:43:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:21.482240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.508979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d3b46-f484-4f25-bf07-fda7a4d8eff5 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.509521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d3b46-f484-4f25-bf07-fda7a4d8eff5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.509804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d3b46-f484-4f25-bf07-fda7a4d8eff5 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.511891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 527/1040] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:21.517658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3317-9dca-437a-a2eb-5dfc4d6156be None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.518242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3317-9dca-437a-a2eb-5dfc4d6156be None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.518690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3317-9dca-437a-a2eb-5dfc4d6156be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.521797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:21.543499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-430e3317-9dca-437a-a2eb-5dfc4d6156be None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:21.544744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-430e3317-9dca-437a-a2eb-5dfc4d6156be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:21.592216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e42690-c9b9-4988-b25c-870c6cb20bda None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:21.617803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 528/1041] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:21.630941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:21.631555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.631833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.632104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.640938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3317-9dca-437a-a2eb-5dfc4d6156be None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.708959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:21.710421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:21.743254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:21.747564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3317-9dca-437a-a2eb-5dfc4d6156be None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:21.750251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:21.786071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:21.786623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.786896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.787162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.799196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:21.803896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:21.804999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:21.805842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:21.810020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 529/1042] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:21.810799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.826780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 514/1043] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:21.832347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.913046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da30762-713d-402d-a5a5-58d74f2eb303 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:21.937043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdacc62-6ec9-470e-a926-335f6d33ca68 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:21.947842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:21.948482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.948760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.949038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.949690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:21.950107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:21.951458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:21.960239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 530/1044] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:43:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:21.966236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:21.975247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:21.975892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:21.976189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:21.976470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:21.979185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:21.991645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:21.993738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:21.995276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:22.010500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:22.011494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:22.013062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:22.013062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:22.013062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:22.013062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 515/1045] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:22.017346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.077798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd08880-3915-41c1-bcc4-0c1cef6a118b None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.115329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.116438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.116720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.117150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] PATH_INFO: `/v3/users/cb6ae783d2f34501b103e9e95586dd06` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.127593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd520b94-25f9-448c-b8e0-82cdacfd482a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.132298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] RBAC: Authorizing `identity:get_user(user_id=cb6ae783d2f34501b103e9e95586dd06)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.133774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.141158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 531/1046] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:21 2022] GET /identity/v3/users/cb6ae783d2f34501b103e9e95586dd06 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:22.142976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.164390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.165004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.165288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.165557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.166196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:22.166498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.167871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.180886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 516/1047] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:22.186074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.233932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:22.241976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:22.259539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa542ab-9755-4d07-a861-e3608d9b896e None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.298292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.298923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.299201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.299475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.299582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa14d31-2aa8-4458-be3c-0f7491c2b6d5 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.336424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.336990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.337260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.337532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.345186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.346638 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:43:22.358052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:22.358995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:22.359853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:22.364395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 517/1048] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:22.365086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.376921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.378378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.471054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:22.479272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828fcc0-046c-4691-882f-3d3a31f78d5c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.489990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:22.496480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 532/1049] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 16:43:22.503081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.518399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.519380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.519867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.520263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.520967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:22.521371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.522917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.541250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 518/1050] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:22.542175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.611298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc7c0d-72d5-4fc5-933b-761dfa54cb29 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.649929 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09e419-ded3-449d-94ec-647e23333503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.650565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09e419-ded3-449d-94ec-647e23333503 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.650840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09e419-ded3-449d-94ec-647e23333503 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.651110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09e419-ded3-449d-94ec-647e23333503 None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.651648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c072-fb88-416f-a85c-bc903ba7dee6 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.652261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09e419-ded3-449d-94ec-647e23333503 None admin] RBAC: Authorizing `identity:get_project(project_id=316cba77e4564d5395af74748cd0a3d3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.653684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09e419-ded3-449d-94ec-647e23333503 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.660630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 533/1051] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:22.660630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11494-0847-481c-ad53-ca048818a354 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.660630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11494-0847-481c-ad53-ca048818a354 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.660630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11494-0847-481c-ad53-ca048818a354 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.660630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 534/1052] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:22.661606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11494-0847-481c-ad53-ca048818a354 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.689408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a327e3e-73d2-4011-b69d-537788c11963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.690073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a327e3e-73d2-4011-b69d-537788c11963 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.690348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a327e3e-73d2-4011-b69d-537788c11963 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.690634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a327e3e-73d2-4011-b69d-537788c11963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.691608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a327e3e-73d2-4011-b69d-537788c11963 None admin] RBAC: Adding query filter params (role.id=457c69fdf6cc4e33885c6cc4bac6b9ec, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=86456c6dbe4e454e9f255960b70c0832) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:22.691909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a327e3e-73d2-4011-b69d-537788c11963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.693364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a327e3e-73d2-4011-b69d-537788c11963 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.709788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 519/1053] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/role_assignments?user.id=86456c6dbe4e454e9f255960b70c0832&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=457c69fdf6cc4e33885c6cc4bac6b9ec => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:22.709788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036e667-c8f5-4a15-8905-926619fdee38 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.709788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036e667-c8f5-4a15-8905-926619fdee38 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.709788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036e667-c8f5-4a15-8905-926619fdee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.715127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:22.727483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c036e667-c8f5-4a15-8905-926619fdee38 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:22.729027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c036e667-c8f5-4a15-8905-926619fdee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:22.758292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:22.782730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11494-0847-481c-ad53-ca048818a354 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.804912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 520/1054] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:22.821951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.822581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.822859 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.823130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.823879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=316cba77e4564d5395af74748cd0a3d3, user.id=cb6ae783d2f34501b103e9e95586dd06) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:22.824222 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.825694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:22.828976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036e667-c8f5-4a15-8905-926619fdee38 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:22.832497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 535/1055] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/role_assignments?user.id=cb6ae783d2f34501b103e9e95586dd06&scope.project.id=316cba77e4564d5395af74748cd0a3d3&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:22.949405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036e667-c8f5-4a15-8905-926619fdee38 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:22.989178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:22.989829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:22.990106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:22.990372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] PATH_INFO: `/v3/services/image` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:22.990955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:22.992893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:23.003570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:23.004865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:23.005727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 521/1056] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Aug 10 16:43:22 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:23.010737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:23.125537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d46cd-60ab-47ff-87a0-89bc12d5e414 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:23.136646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:23.162854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:23.163481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:23.163762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:23.164033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:23.164706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] RBAC: Adding query filter params (name=image) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:23.165014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:23.166385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:23.177107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 536/1057] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:23.178285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:23.209476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:23.287647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f2a77-021b-4ad6-8718-031023d56abe None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:23.323928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101bbb4-2b0b-4654-9a16-b5328b36e01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:23.324573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101bbb4-2b0b-4654-9a16-b5328b36e01a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:23.324851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101bbb4-2b0b-4654-9a16-b5328b36e01a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:23.325123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101bbb4-2b0b-4654-9a16-b5328b36e01a None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:23.325774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101bbb4-2b0b-4654-9a16-b5328b36e01a None admin] RBAC: Adding query filter params (type=image) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:23.326077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101bbb4-2b0b-4654-9a16-b5328b36e01a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:23.327439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101bbb4-2b0b-4654-9a16-b5328b36e01a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:23.337126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 522/1058] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:23.338382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:23.446939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f835e1-0c40-46d0-9b95-b99c9b858d98 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:23.483483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695cbb-808b-49f3-8a3b-a311bbf30b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:23.484681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695cbb-808b-49f3-8a3b-a311bbf30b71 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:23.484960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695cbb-808b-49f3-8a3b-a311bbf30b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:23.485232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695cbb-808b-49f3-8a3b-a311bbf30b71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:23.485915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7695cbb-808b-49f3-8a3b-a311bbf30b71 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:23.487290 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7695cbb-808b-49f3-8a3b-a311bbf30b71 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:23.511989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 537/1059] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:43:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:43:24.821115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f94e46-0c7c-4dc0-89f8-90c743fb5655 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:24.821710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f94e46-0c7c-4dc0-89f8-90c743fb5655 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:24.821710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f94e46-0c7c-4dc0-89f8-90c743fb5655 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:24.823026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 523/1060] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:24.828794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e3b4a-5fc6-446a-bd94-57c6daddd391 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:24.829189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e3b4a-5fc6-446a-bd94-57c6daddd391 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:24.829380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e3b4a-5fc6-446a-bd94-57c6daddd391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:24.847480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c98e3b4a-5fc6-446a-bd94-57c6daddd391 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:24.848769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c98e3b4a-5fc6-446a-bd94-57c6daddd391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:24.856786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acd8f-6c82-4d28-adc7-a94b1654ed5e None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:24.857076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acd8f-6c82-4d28-adc7-a94b1654ed5e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:24.857352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acd8f-6c82-4d28-adc7-a94b1654ed5e None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:24.859030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 524/1061] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:24.863742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f6ed1-6557-4edd-98d6-30d334c2c93f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:24.864027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f6ed1-6557-4edd-98d6-30d334c2c93f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:24.864340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f6ed1-6557-4edd-98d6-30d334c2c93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:24.877594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:24.883449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-806f6ed1-6557-4edd-98d6-30d334c2c93f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:24.884842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-806f6ed1-6557-4edd-98d6-30d334c2c93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:24.931165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 538/1062] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:24.956367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e3b4a-5fc6-446a-bd94-57c6daddd391 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:24.964733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 525/1063] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:24.987806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f6ed1-6557-4edd-98d6-30d334c2c93f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.011746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.069806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e3b4a-5fc6-446a-bd94-57c6daddd391 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.097036 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f6ed1-6557-4edd-98d6-30d334c2c93f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.106847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.107482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.107877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.108164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.119232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:25.124993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:25.126078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:25.126923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 539/1064] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 10 16:43:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:25.132298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.135230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.135874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.136169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.136456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.151153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:25.153459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:25.159614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:25.160590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:25.161394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 526/1065] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:43:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:25.167773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.259879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ce4ae-5755-4971-8f45-2f45582090c8 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.291382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.304480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.305360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.305770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.306183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.307202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:25.307701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.309302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.313907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3b753-5cd4-412e-86f3-ec3bf17fed14 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.323716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 540/1066] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:25.324875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.356209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849a436-b199-460e-b8f3-128041180631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.356866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849a436-b199-460e-b8f3-128041180631 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.357226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849a436-b199-460e-b8f3-128041180631 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.357675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849a436-b199-460e-b8f3-128041180631 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.358430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849a436-b199-460e-b8f3-128041180631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:25.359276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849a436-b199-460e-b8f3-128041180631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.360876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849a436-b199-460e-b8f3-128041180631 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.369926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 527/1067] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:25.376769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849a436-b199-460e-b8f3-128041180631 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.412139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.443425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06d431-dae9-40d5-9ce2-652c991ba59f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.489584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.490232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.490514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.490789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.495224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849a436-b199-460e-b8f3-128041180631 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.502980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.504480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:25.519318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:25.520554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:25.521438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:25.521438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:25.521438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:25.521438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 541/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:25.525871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.538363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.539012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.539289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.539888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] PATH_INFO: `/v3/users/cb6ae783d2f34501b103e9e95586dd06` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.541088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] RBAC: Authorizing `identity:get_user(user_id=cb6ae783d2f34501b103e9e95586dd06)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.542529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.544529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 528/1069] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/users/cb6ae783d2f34501b103e9e95586dd06 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:25.551919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.567432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.644832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae87b2-cd71-4611-81aa-aae8291c99a2 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.671183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03c08d-962e-4c49-a4ec-5d64b2252570 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.674459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.685765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ac7f8-a2bd-452d-876f-043e0060d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.686383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ac7f8-a2bd-452d-876f-043e0060d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.686659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ac7f8-a2bd-452d-876f-043e0060d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.686933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ac7f8-a2bd-452d-876f-043e0060d4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.687578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ac7f8-a2bd-452d-876f-043e0060d4f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:25.687880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ac7f8-a2bd-452d-876f-043e0060d4f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.689286 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ac7f8-a2bd-452d-876f-043e0060d4f7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.701943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 542/1070] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:25.703171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723472c-2b7f-4956-9fed-8c3838bce62e None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.703455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723472c-2b7f-4956-9fed-8c3838bce62e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.703726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723472c-2b7f-4956-9fed-8c3838bce62e None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.704959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 543/1071] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:25.707856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723472c-2b7f-4956-9fed-8c3838bce62e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.713924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1332e8-03d2-49d4-abab-6420dfb74044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.714561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1332e8-03d2-49d4-abab-6420dfb74044 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.714839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1332e8-03d2-49d4-abab-6420dfb74044 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.715112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1332e8-03d2-49d4-abab-6420dfb74044 None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.716261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1332e8-03d2-49d4-abab-6420dfb74044 None admin] RBAC: Authorizing `identity:get_project(project_id=316cba77e4564d5395af74748cd0a3d3)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.717693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1332e8-03d2-49d4-abab-6420dfb74044 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.721472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 529/1072] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:25.721472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca0701-30d9-4e67-b5e6-ce629545b62c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.721472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca0701-30d9-4e67-b5e6-ce629545b62c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.721472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca0701-30d9-4e67-b5e6-ce629545b62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.740707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-eeca0701-30d9-4e67-b5e6-ce629545b62c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:25.741897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-eeca0701-30d9-4e67-b5e6-ce629545b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:25.811298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 530/1073] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:25.812174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca0701-30d9-4e67-b5e6-ce629545b62c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.815479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723472c-2b7f-4956-9fed-8c3838bce62e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.825346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.854356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.854997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.855285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.855554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.863675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.865216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:43:25.876562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:25.877564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 544/1074] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:25.879058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.921824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca0701-30d9-4e67-b5e6-ce629545b62c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:25.929125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:25.956055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:25.956874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:25.957151 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:25.957428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3/users/cb6ae783d2f34501b103e9e95586dd06/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:25.959346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] RBAC: Authorizing `identity:create_grant(project_id=316cba77e4564d5395af74748cd0a3d3, user_id=cb6ae783d2f34501b103e9e95586dd06, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:25.960815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:25.974873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 531/1075] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:43:25 2022] PUT /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3/users/cb6ae783d2f34501b103e9e95586dd06/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:43:25.977201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:25.984533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8347b-a7b0-4c34-ba7f-5da9f5de7dff None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.025572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.026562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.026841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.027129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.059301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:26.106326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44af226-8857-47ae-a5ff-93f44ee460ba None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.112446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.113956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.151720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.152375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.152661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.152981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.153776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:26.154079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.156166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.177032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:26.180319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 532/1076] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:26.185991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:26.233708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 545/1077] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:26.239487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:26.301744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f92b9-998e-4058-9c86-81b11bea6b6b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.330657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:26.341380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.342017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.342299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.342571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.347468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bf548-b915-4f80-ac35-57a4a5abe333 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.353220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.354662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:26.371701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:26.372704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:26.373593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:26.373593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:26.373593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:26.374690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 533/1078] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:26.379765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:26.385980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.386867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.387323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.387769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.400883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.402690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:26.419185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:26.420272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:26.421198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:26.421198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:26.421198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:26.421198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 546/1079] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:26.426569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:26.429759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:26.498162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25c9b5-abb8-4767-9d01-221dd4a6816a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.536139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.536782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.537060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.537333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.537971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:26.538272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.539637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.544330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06cd09-b8bf-4536-b367-60da9b0c1e29 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.552901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 534/1080] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:26.558522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:26.585149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.585896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.586179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.586453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.587101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:26.587428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.588834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.601962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 547/1081] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:26.613827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:26.683872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83189eb-0c6d-4e54-aa25-a35a9a59b4f8 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.724436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.725397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.725834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.726255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.734335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.736069 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:43:26.749271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:26.750312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:26.751195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 535/1082] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:26.757286 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:26.828927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:26.870596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce2d5e-49b3-4665-9c11-58d6db232b56 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:26.908284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce5a1-00f9-4ba9-a131-d7baab6c04d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:26.908926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce5a1-00f9-4ba9-a131-d7baab6c04d8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:26.909206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce5a1-00f9-4ba9-a131-d7baab6c04d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:26.909479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce5a1-00f9-4ba9-a131-d7baab6c04d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:26.910173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce5a1-00f9-4ba9-a131-d7baab6c04d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:26.910506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce5a1-00f9-4ba9-a131-d7baab6c04d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:26.911937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce5a1-00f9-4ba9-a131-d7baab6c04d8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:26.923842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 548/1083] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:26.930303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:27.093875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:27.099423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6990f-394f-4bf7-ad45-1a5116750078 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:27.135512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7b7f6-050b-4eb3-9a05-f846bc12a5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:27.136206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7b7f6-050b-4eb3-9a05-f846bc12a5cd None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:27.136490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7b7f6-050b-4eb3-9a05-f846bc12a5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:27.136764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7b7f6-050b-4eb3-9a05-f846bc12a5cd None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/86456c6dbe4e454e9f255960b70c0832/roles/457c69fdf6cc4e33885c6cc4bac6b9ec` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:27.138988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7b7f6-050b-4eb3-9a05-f846bc12a5cd None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=86456c6dbe4e454e9f255960b70c0832, role_id=457c69fdf6cc4e33885c6cc4bac6b9ec)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:27.141158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7b7f6-050b-4eb3-9a05-f846bc12a5cd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:27.154192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 536/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:43:26 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/86456c6dbe4e454e9f255960b70c0832/roles/457c69fdf6cc4e33885c6cc4bac6b9ec => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:43:28.000843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea928e6-e28b-437d-ac00-5d29420e16e4 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:28.001367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea928e6-e28b-437d-ac00-5d29420e16e4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:28.001367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea928e6-e28b-437d-ac00-5d29420e16e4 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:28.002733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 549/1085] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:28.008137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917677-9b43-41eb-b544-47482cb715b5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:28.008572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917677-9b43-41eb-b544-47482cb715b5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:28.008960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917677-9b43-41eb-b544-47482cb715b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:28.028516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9b917677-9b43-41eb-b544-47482cb715b5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:28.029714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9b917677-9b43-41eb-b544-47482cb715b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:28.097233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 537/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:28.120862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea928e6-e28b-437d-ac00-5d29420e16e4 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:28.234960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea928e6-e28b-437d-ac00-5d29420e16e4 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:28.258565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:28.270852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:28.271511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:28.271788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:28.272058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:28.283952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:28.288682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:28.289740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:28.290573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 550/1087] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:28.295938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917677-9b43-41eb-b544-47482cb715b5 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:28.383615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:28.396406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917677-9b43-41eb-b544-47482cb715b5 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:28.430588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:28.431193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:28.431469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:28.431737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:28.432424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:28.432723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:28.434087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:28.443536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 538/1088] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:28.449351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:28.553078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8958318-78b9-45e9-87aa-587c96f04d35 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:28.587041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:28.587673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:28.587954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:28.588259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] PATH_INFO: `/v3/users/cb6ae783d2f34501b103e9e95586dd06` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:28.589388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] RBAC: Authorizing `identity:get_user(user_id=cb6ae783d2f34501b103e9e95586dd06)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:28.590824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:28.592626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 551/1089] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:28 2022] GET /identity/v3/users/cb6ae783d2f34501b103e9e95586dd06 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:28.598711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:28.702197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2f4ac-9d0f-406a-9da4-ebfa91a93a34 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:28.737655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:28.738272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:28.738561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:28.738831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:28.758416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:28.808630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:28.809981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:28.918328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 539/1090] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:28.924478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:28.975211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:29.039018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922976b-7779-4035-b2ae-17ffb24ab104 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:29.081261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783eec5-654c-4e71-a977-559518037874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:29.081796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783eec5-654c-4e71-a977-559518037874 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.082075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783eec5-654c-4e71-a977-559518037874 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.082347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783eec5-654c-4e71-a977-559518037874 None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.083450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783eec5-654c-4e71-a977-559518037874 None admin] RBAC: Authorizing `identity:get_project(project_id=316cba77e4564d5395af74748cd0a3d3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:29.084912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783eec5-654c-4e71-a977-559518037874 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:29.086552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 552/1091] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:28 2022] GET /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:29.092707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:29.142244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:29.192215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d129ea-61d5-4422-b42a-d24fc00d1909 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:29.204453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5aa4b-bfa7-4cd6-83de-57ebe43e1bfb None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.204732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5aa4b-bfa7-4cd6-83de-57ebe43e1bfb None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.205005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5aa4b-bfa7-4cd6-83de-57ebe43e1bfb None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.206362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 553/1092] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:29.211170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd515-a082-488a-8afb-5969e60ae37d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.211448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd515-a082-488a-8afb-5969e60ae37d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.211719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd515-a082-488a-8afb-5969e60ae37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.226846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:29.227457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.227741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.228017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.228775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=316cba77e4564d5395af74748cd0a3d3, user.id=cb6ae783d2f34501b103e9e95586dd06) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:29.229084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:29.230515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:29.230722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-1a6fd515-a082-488a-8afb-5969e60ae37d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:29.232056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1a6fd515-a082-488a-8afb-5969e60ae37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:29.243034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 540/1093] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:29 2022] GET /identity/v3/role_assignments?user.id=cb6ae783d2f34501b103e9e95586dd06&scope.project.id=316cba77e4564d5395af74748cd0a3d3&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:29.304474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 554/1094] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:29.329447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:29.436543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f760b-fa04-471f-9e78-9505b0a90a42 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:29.451433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:29.472433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:29.473068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.473348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.473621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.483749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:29.488924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:29.489842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:29.490673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 541/1095] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 10 16:43:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:29.496232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fd515-a082-488a-8afb-5969e60ae37d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:29.545440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:29.598934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fd515-a082-488a-8afb-5969e60ae37d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:29.632692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:29.633308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.633589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.633860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.634519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:29.634823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:29.636206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:29.644705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 555/1096] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 10 16:43:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:29.650542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:29.754505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24c8f-715f-4f5a-87b0-d65f476785aa None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:29.787654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:29.788320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.788600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.788872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.799699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:29.801241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:29.815398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:29.816597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:29.817461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:29.817461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:29.817461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:29.817461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 542/1097] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:29.822511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:29.919012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc2000-b310-4bbc-b17d-3ea840f2d8d6 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:29.929695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:29.953941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:29.954551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:29.954828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:29.955099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:29.955740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:29.956041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:29.957431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:29.969337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 556/1098] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:29.974877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:30.003185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:30.073718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7623af-303e-49a5-ad76-385709c6e985 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:30.105878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:30.106491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:30.106893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:30.107161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:30.114806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:30.116251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:43:30.127054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:30.127961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:30.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 543/1099] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 10 16:43:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:43:30.133865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:30.230318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b8297-87d2-478e-8b86-c753ec2a537d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:30.264133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:30.264779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:30.265055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:30.265319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:30.266030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:30.266333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:30.267737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:30.284827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 557/1100] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 10 16:43:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:43:30.290565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:30.376424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:30.393083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb981bd-c337-4e02-8d0b-99ef29bc5c78 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:30.429632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:30.430242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:30.430522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:30.430793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:30.497542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:30.498986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:30.580754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:30.596724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 544/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:30.602412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:30.631184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:30.700958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257e8b4-945a-482f-8bb6-f9a602fd0f68 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:30.738913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:30.739543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:30.739819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:30.740083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:30.750797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:30.752264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:30.767660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:30.768686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:43:30.769547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:30.769547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:43:30.769547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:30.769547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 558/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:30.774540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:30.870340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccc4b9-a708-4ba8-b669-ec6472d398bb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:30.903920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:30.904429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:30.904704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:30.904971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:30.905633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:30.905932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:30.907276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:30.918209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 545/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:30.924089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:31.010268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:31.030108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d63d6-2134-4756-82b9-ac9bedb66398 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:31.062797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:31.063548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:31.063899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:31.064276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:31.071158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:31.072632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:43:31.084565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:31.085517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:31.086392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 559/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:43:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:31.091549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:31.136969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:31.190019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c93166-adb8-4721-823d-f4f38f6aac99 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:31.222361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa655b-427f-4f7c-9c26-efeca01bdd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:31.222968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa655b-427f-4f7c-9c26-efeca01bdd00 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:31.223245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa655b-427f-4f7c-9c26-efeca01bdd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:31.223514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa655b-427f-4f7c-9c26-efeca01bdd00 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:31.224233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa655b-427f-4f7c-9c26-efeca01bdd00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:31.224537 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa655b-427f-4f7c-9c26-efeca01bdd00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:31.225953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa655b-427f-4f7c-9c26-efeca01bdd00 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:31.238823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 546/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:43:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:31.245258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:31.296508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f9d00-f6b9-4fc3-ac23-70f3e71baf64 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:31.296792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f9d00-f6b9-4fc3-ac23-70f3e71baf64 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:31.297073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f9d00-f6b9-4fc3-ac23-70f3e71baf64 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:31.298407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 547/1106] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:43:31.303379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418029ea-98fd-4b20-98c0-93e95b6daa5e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:31.303662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418029ea-98fd-4b20-98c0-93e95b6daa5e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:31.303941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418029ea-98fd-4b20-98c0-93e95b6daa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:31.322185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-418029ea-98fd-4b20-98c0-93e95b6daa5e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:31.323584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-418029ea-98fd-4b20-98c0-93e95b6daa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:31.345380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f58ed-241f-4d7b-9bcf-90e33da1f598 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:31.379887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:31.380524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:31.380803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:31.381077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:31.381782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] RBAC: Adding query filter params (role.id=457c69fdf6cc4e33885c6cc4bac6b9ec, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=86456c6dbe4e454e9f255960b70c0832) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:31.382088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:31.383499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:31.389198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 560/1107] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:31 2022] GET /identity/v3/role_assignments?user.id=86456c6dbe4e454e9f255960b70c0832&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=457c69fdf6cc4e33885c6cc4bac6b9ec => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:31.390760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 548/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:31.415198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:31.419230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:31.529637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a4228-4acf-4c7d-8c7b-3e4cb5b36de8 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:31.565922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d314191-737d-4e05-80b0-82ce5c1ede8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:31.566802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314191-737d-4e05-80b0-82ce5c1ede8a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:31.567077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314191-737d-4e05-80b0-82ce5c1ede8a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:31.567345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314191-737d-4e05-80b0-82ce5c1ede8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:31.568503 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d314191-737d-4e05-80b0-82ce5c1ede8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:31.569921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d314191-737d-4e05-80b0-82ce5c1ede8a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:31.571475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 561/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:31.577399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418029ea-98fd-4b20-98c0-93e95b6daa5e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:31.616241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:31.682651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418029ea-98fd-4b20-98c0-93e95b6daa5e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:31.717243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c0443-6f0f-4eee-8f40-c3cc9b06d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:31.718365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c0443-6f0f-4eee-8f40-c3cc9b06d41f None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:31.718642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c0443-6f0f-4eee-8f40-c3cc9b06d41f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:31.718914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c0443-6f0f-4eee-8f40-c3cc9b06d41f None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:31.719560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c0443-6f0f-4eee-8f40-c3cc9b06d41f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:31.721043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c0443-6f0f-4eee-8f40-c3cc9b06d41f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:31.744904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 549/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:43:31 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:33.812537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6b25f-c827-4ce9-a361-1d01ee017dc6 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:33.813097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6b25f-c827-4ce9-a361-1d01ee017dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:33.813097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6b25f-c827-4ce9-a361-1d01ee017dc6 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:33.814424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 562/1111] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:33.819725 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a023-8633-41a3-a5b4-287f528d6332 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:33.820110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a023-8633-41a3-a5b4-287f528d6332 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:33.820329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a023-8633-41a3-a5b4-287f528d6332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:33.838639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-34e4a023-8633-41a3-a5b4-287f528d6332 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:33.839832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-34e4a023-8633-41a3-a5b4-287f528d6332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:33.904921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 550/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:33.927469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6b25f-c827-4ce9-a361-1d01ee017dc6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:34.000432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:34.035922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6b25f-c827-4ce9-a361-1d01ee017dc6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:34.069570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:34.070142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:34.070423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:34.070695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] PATH_INFO: `/v3/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:34.072096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] RBAC: Authorizing `identity:get_role(role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:34.073492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:34.074814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 563/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:33 2022] GET /identity/v3/roles/491f8de5da624289a3a9b28c155d764c => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:34.080672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a023-8633-41a3-a5b4-287f528d6332 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:34.104372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:34.179666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a023-8633-41a3-a5b4-287f528d6332 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:34.213025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:34.213647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:34.213925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:34.214196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] PATH_INFO: `/v3/users/af7a16e3eefb4511b50300f6568b7612` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:34.227453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] RBAC: Authorizing `identity:get_user(user_id=af7a16e3eefb4511b50300f6568b7612)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:34.228904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:34.230487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 551/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:34 2022] GET /identity/v3/users/af7a16e3eefb4511b50300f6568b7612 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:34.236266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:34.334895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca07dce-4eb4-4b22-986d-d4083f354d33 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:34.367008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:34.367606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:34.367899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:34.368168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:34.379009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:34.437715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:34.439192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:34.534747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 564/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:34.540652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:34.620223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:34.639565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5700a51-45fa-4bac-aee8-8a3f658748bf None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:34.673409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24248589-a6cf-4e12-9535-6ce83d588f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:34.674029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248589-a6cf-4e12-9535-6ce83d588f64 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:34.674306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248589-a6cf-4e12-9535-6ce83d588f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:34.674576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248589-a6cf-4e12-9535-6ce83d588f64 None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:34.675684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24248589-a6cf-4e12-9535-6ce83d588f64 None admin] RBAC: Authorizing `identity:get_project(project_id=316cba77e4564d5395af74748cd0a3d3)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:34.677141 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24248589-a6cf-4e12-9535-6ce83d588f64 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:34.678763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 552/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:34 2022] GET /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:34.684945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:34.724936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:34.782604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e98829-8f2a-4620-a4b8-399d7e59df11 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:34.815245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f665463-7c2d-4baf-aee9-3a270c4f3e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:34.815837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f665463-7c2d-4baf-aee9-3a270c4f3e1f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:34.816114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f665463-7c2d-4baf-aee9-3a270c4f3e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:34.816682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f665463-7c2d-4baf-aee9-3a270c4f3e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:34.817391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f665463-7c2d-4baf-aee9-3a270c4f3e1f None admin] RBAC: Adding query filter params (role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=316cba77e4564d5395af74748cd0a3d3, user.id=af7a16e3eefb4511b50300f6568b7612) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:34.817697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f665463-7c2d-4baf-aee9-3a270c4f3e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:34.819101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f665463-7c2d-4baf-aee9-3a270c4f3e1f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:34.824257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 565/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:34 2022] GET /identity/v3/role_assignments?user.id=af7a16e3eefb4511b50300f6568b7612&scope.project.id=316cba77e4564d5395af74748cd0a3d3&role.id=491f8de5da624289a3a9b28c155d764c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:36.778609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc302-837d-4a1d-81ac-db39d1296204 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:36.779168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc302-837d-4a1d-81ac-db39d1296204 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:36.779168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc302-837d-4a1d-81ac-db39d1296204 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:36.781107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 553/1118] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:36.786453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe335cc-0f0f-4a52-9939-1aaad4fab1e4 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:36.786873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe335cc-0f0f-4a52-9939-1aaad4fab1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:36.787002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe335cc-0f0f-4a52-9939-1aaad4fab1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:36.809236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0fe335cc-0f0f-4a52-9939-1aaad4fab1e4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:36.810405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0fe335cc-0f0f-4a52-9939-1aaad4fab1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:36.873938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 566/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:43:36.895684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc302-837d-4a1d-81ac-db39d1296204 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:37.000919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc302-837d-4a1d-81ac-db39d1296204 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:37.033889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:37.034450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:37.034728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:37.034996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] PATH_INFO: `/v3/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:37.036412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] RBAC: Authorizing `identity:get_role(role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:37.037778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:37.039142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 554/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:36 2022] GET /identity/v3/roles/491f8de5da624289a3a9b28c155d764c => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:37.045075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe335cc-0f0f-4a52-9939-1aaad4fab1e4 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:37.081411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:37.147674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe335cc-0f0f-4a52-9939-1aaad4fab1e4 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:37.180506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:37.181161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:37.181440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:37.181713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] PATH_INFO: `/v3/users/af7a16e3eefb4511b50300f6568b7612` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:37.182826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] RBAC: Authorizing `identity:get_user(user_id=af7a16e3eefb4511b50300f6568b7612)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:37.184274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:37.185845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 567/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:37 2022] GET /identity/v3/users/af7a16e3eefb4511b50300f6568b7612 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:37.191926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:37.210211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:37.291988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc5f8-df10-47c0-b1cc-03fc5201387d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:37.325158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61690a-63b7-4257-ba6b-77817367c656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:37.325771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61690a-63b7-4257-ba6b-77817367c656 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:37.326054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61690a-63b7-4257-ba6b-77817367c656 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:37.326330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61690a-63b7-4257-ba6b-77817367c656 None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:37.327969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61690a-63b7-4257-ba6b-77817367c656 None admin] RBAC: Authorizing `identity:get_project(project_id=316cba77e4564d5395af74748cd0a3d3)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:37.329423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61690a-63b7-4257-ba6b-77817367c656 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:37.331043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 555/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:37 2022] GET /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:37.337024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:37.435147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc2486-5bf8-47e5-9a9d-42ba1a0cdd95 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:37.436737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:37.470804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421f7f2-c332-4326-938f-d0d9af46b2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:37.471452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421f7f2-c332-4326-938f-d0d9af46b2c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:37.471732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421f7f2-c332-4326-938f-d0d9af46b2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:37.472005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421f7f2-c332-4326-938f-d0d9af46b2c4 None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3/users/af7a16e3eefb4511b50300f6568b7612/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:37.474163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421f7f2-c332-4326-938f-d0d9af46b2c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=316cba77e4564d5395af74748cd0a3d3, user_id=af7a16e3eefb4511b50300f6568b7612, role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:37.475602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421f7f2-c332-4326-938f-d0d9af46b2c4 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:37.487656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 568/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:43:37 2022] PUT /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3/users/af7a16e3eefb4511b50300f6568b7612/roles/491f8de5da624289a3a9b28c155d764c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:43:39.447659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127d684-e28b-416a-8763-d0cc531d8f8f None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:39.448224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127d684-e28b-416a-8763-d0cc531d8f8f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:39.448224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127d684-e28b-416a-8763-d0cc531d8f8f None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:39.449561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 556/1124] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:39.455430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0380fde-5649-4a02-90a5-f7c16f09c16c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:39.455883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0380fde-5649-4a02-90a5-f7c16f09c16c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:39.455984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0380fde-5649-4a02-90a5-f7c16f09c16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:39.474219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c0380fde-5649-4a02-90a5-f7c16f09c16c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:39.475404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c0380fde-5649-4a02-90a5-f7c16f09c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:39.539562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 569/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:39.563471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127d684-e28b-416a-8763-d0cc531d8f8f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:39.656792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:39.674636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127d684-e28b-416a-8763-d0cc531d8f8f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:39.708796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:39.709419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:39.709697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:39.709970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] PATH_INFO: `/v3/roles/491f8de5da624289a3a9b28c155d764c` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:39.711369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] RBAC: Authorizing `identity:get_role(role_id=491f8de5da624289a3a9b28c155d764c)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:39.712771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:39.714085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 557/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:39 2022] GET /identity/v3/roles/491f8de5da624289a3a9b28c155d764c => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:39.719931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0380fde-5649-4a02-90a5-f7c16f09c16c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:39.808539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:39.819977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0380fde-5649-4a02-90a5-f7c16f09c16c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:39.862180 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:39.862800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:39.863080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:39.863350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] PATH_INFO: `/v3/users/af7a16e3eefb4511b50300f6568b7612` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:39.864481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] RBAC: Authorizing `identity:get_user(user_id=af7a16e3eefb4511b50300f6568b7612)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:39.865901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:39.867460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 570/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:39 2022] GET /identity/v3/users/af7a16e3eefb4511b50300f6568b7612 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:39.873338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:39.969718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba4a64-c653-4474-a413-e50604449f94 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:40.003074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:40.003662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:40.003941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:40.004240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:40.024441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:40.069626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:40.071097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:40.165672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 558/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:40.171537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:40.269937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e69af-819f-47e9-94bf-ebd1b39638ec None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:40.304193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36485a-d234-40dc-9c28-414e791cfd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:40.304823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36485a-d234-40dc-9c28-414e791cfd7b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:40.305098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36485a-d234-40dc-9c28-414e791cfd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:40.305363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36485a-d234-40dc-9c28-414e791cfd7b None admin] PATH_INFO: `/v3/projects/316cba77e4564d5395af74748cd0a3d3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:40.306473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36485a-d234-40dc-9c28-414e791cfd7b None admin] RBAC: Authorizing `identity:get_project(project_id=316cba77e4564d5395af74748cd0a3d3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:40.307891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36485a-d234-40dc-9c28-414e791cfd7b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:40.309876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 571/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:40 2022] GET /identity/v3/projects/316cba77e4564d5395af74748cd0a3d3 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:40.316172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:40.374285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:40.426248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b75f35-ad00-4892-90d3-eb1e1eb15659 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:40.462076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8158730-9381-4f4d-ab58-f3e05b6c34f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:40.462755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8158730-9381-4f4d-ab58-f3e05b6c34f8 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:40.462932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8158730-9381-4f4d-ab58-f3e05b6c34f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:40.463197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8158730-9381-4f4d-ab58-f3e05b6c34f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:40.463907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8158730-9381-4f4d-ab58-f3e05b6c34f8 None admin] RBAC: Adding query filter params (role.id=491f8de5da624289a3a9b28c155d764c, scope.project.id=316cba77e4564d5395af74748cd0a3d3, user.id=af7a16e3eefb4511b50300f6568b7612) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:40.464247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8158730-9381-4f4d-ab58-f3e05b6c34f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:40.465717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8158730-9381-4f4d-ab58-f3e05b6c34f8 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:40.473004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 559/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:40 2022] GET /identity/v3/role_assignments?user.id=af7a16e3eefb4511b50300f6568b7612&scope.project.id=316cba77e4564d5395af74748cd0a3d3&role.id=491f8de5da624289a3a9b28c155d764c => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:42.509567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ace644-9c39-4b5a-bfab-6ae15eb90020 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:42.510115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ace644-9c39-4b5a-bfab-6ae15eb90020 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:42.510115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ace644-9c39-4b5a-bfab-6ae15eb90020 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:42.511459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 572/1131] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:42.517054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf3f3-9c36-4883-a976-2be686af8ed6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:42.517474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf3f3-9c36-4883-a976-2be686af8ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:42.517585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf3f3-9c36-4883-a976-2be686af8ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:42.535790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-be5cf3f3-9c36-4883-a976-2be686af8ed6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:42.537007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-be5cf3f3-9c36-4883-a976-2be686af8ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:42.602502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 560/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:42.625232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ace644-9c39-4b5a-bfab-6ae15eb90020 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:42.653880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:42.732297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ace644-9c39-4b5a-bfab-6ae15eb90020 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:42.766049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad421d-ba47-49e6-baf8-92f2d5b4dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:42.766669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad421d-ba47-49e6-baf8-92f2d5b4dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:42.766945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad421d-ba47-49e6-baf8-92f2d5b4dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:42.767214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad421d-ba47-49e6-baf8-92f2d5b4dfa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:42.768334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad421d-ba47-49e6-baf8-92f2d5b4dfa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:42.769749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad421d-ba47-49e6-baf8-92f2d5b4dfa0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:42.771300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 573/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:42.777175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf3f3-9c36-4883-a976-2be686af8ed6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:42.823178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:42.875290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf3f3-9c36-4883-a976-2be686af8ed6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:42.909661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0fe4f-53f0-4975-b2d9-ca8f60ea06a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:42.910268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0fe4f-53f0-4975-b2d9-ca8f60ea06a7 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:42.910546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0fe4f-53f0-4975-b2d9-ca8f60ea06a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:42.910814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0fe4f-53f0-4975-b2d9-ca8f60ea06a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:42.911451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0fe4f-53f0-4975-b2d9-ca8f60ea06a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:42.912910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0fe4f-53f0-4975-b2d9-ca8f60ea06a7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:42.941106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 561/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:43:42 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:43:44.997001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61614c6-bc53-450a-a3c7-762a35d59d02 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:44.997528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61614c6-bc53-450a-a3c7-762a35d59d02 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:44.997528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61614c6-bc53-450a-a3c7-762a35d59d02 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:44.999490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 574/1135] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:45.004621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e1920-ceab-4457-9aae-f4bf6e4ec36a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:45.004992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e1920-ceab-4457-9aae-f4bf6e4ec36a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:45.005183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e1920-ceab-4457-9aae-f4bf6e4ec36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:45.022835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-482e1920-ceab-4457-9aae-f4bf6e4ec36a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:45.024023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-482e1920-ceab-4457-9aae-f4bf6e4ec36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:45.090604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 562/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:45.116439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61614c6-bc53-450a-a3c7-762a35d59d02 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:45.207435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:45.214108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61614c6-bc53-450a-a3c7-762a35d59d02 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:45.247101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b711e22-fa5f-4fe8-affc-17210e0771cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:45.247691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711e22-fa5f-4fe8-affc-17210e0771cb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:45.247970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711e22-fa5f-4fe8-affc-17210e0771cb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:45.248263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711e22-fa5f-4fe8-affc-17210e0771cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:45.249417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b711e22-fa5f-4fe8-affc-17210e0771cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:45.250839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b711e22-fa5f-4fe8-affc-17210e0771cb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:45.252429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 575/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:43:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:45.258165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e1920-ceab-4457-9aae-f4bf6e4ec36a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:45.285329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:45.356154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e1920-ceab-4457-9aae-f4bf6e4ec36a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:45.390115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4ec97-c818-460a-830d-724abf6c5888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:45.390712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ec97-c818-460a-830d-724abf6c5888 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:45.390989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ec97-c818-460a-830d-724abf6c5888 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:45.391256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ec97-c818-460a-830d-724abf6c5888 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:45.391898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4ec97-c818-460a-830d-724abf6c5888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:45.393356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4ec97-c818-460a-830d-724abf6c5888 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:45.415587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 563/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:43:45 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:43:47.421975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced311f-045e-4e70-92f8-9e91a426099f None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:47.422430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced311f-045e-4e70-92f8-9e91a426099f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:47.422566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced311f-045e-4e70-92f8-9e91a426099f None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:47.423926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 576/1139] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:47.429182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20160bde-501f-407d-bcfa-39e55d130a2e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:47.429612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20160bde-501f-407d-bcfa-39e55d130a2e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:47.429725 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20160bde-501f-407d-bcfa-39e55d130a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:47.447431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-20160bde-501f-407d-bcfa-39e55d130a2e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:47.448995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-20160bde-501f-407d-bcfa-39e55d130a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:47.513498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 564/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:47.547012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced311f-045e-4e70-92f8-9e91a426099f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:47.647867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced311f-045e-4e70-92f8-9e91a426099f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:47.681270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:47.681897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:47.682175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:47.682446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:47.692576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:47.697216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:47.698348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:47.699279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 577/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:47.704949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20160bde-501f-407d-bcfa-39e55d130a2e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:47.721127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:47.804372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20160bde-501f-407d-bcfa-39e55d130a2e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:47.837448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:47.838042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:47.838318 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:47.838584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:47.839232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:47.839527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:47.840912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:47.849196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 565/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:47.855024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:47.888891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:47.962625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4965b7-8621-4c28-b7d9-da90a66dcb28 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:47.995524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:47.996171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:47.996455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:47.996723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] PATH_INFO: `/v3/users/30dccb7e6b444fa294e3525c0876d5c4` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:48.009717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] RBAC: Authorizing `identity:get_user(user_id=30dccb7e6b444fa294e3525c0876d5c4)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:48.011149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:48.012890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 578/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:47 2022] GET /identity/v3/users/30dccb7e6b444fa294e3525c0876d5c4 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:48.019253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:48.093924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:48.117159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e4c6-7c21-4061-b2ff-6237a60687e3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:48.149925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:48.150515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:48.150790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:48.151055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:48.215897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:48.217324 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:48.288565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:48.312028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 566/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:48.318215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:48.397379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:48.415424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb188-ebdc-453c-9689-f4afc3293517 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:48.450130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e90f7-093d-4063-b2ea-62c56ba7992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:48.450759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e90f7-093d-4063-b2ea-62c56ba7992b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:48.451040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e90f7-093d-4063-b2ea-62c56ba7992b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:48.451314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e90f7-093d-4063-b2ea-62c56ba7992b None admin] PATH_INFO: `/v3/projects/4cbb57adaf32457698920e6dabd42681` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:48.452451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e90f7-093d-4063-b2ea-62c56ba7992b None admin] RBAC: Authorizing `identity:get_project(project_id=4cbb57adaf32457698920e6dabd42681)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:48.453875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e90f7-093d-4063-b2ea-62c56ba7992b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:48.455502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 579/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:48 2022] GET /identity/v3/projects/4cbb57adaf32457698920e6dabd42681 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:48.461716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:48.556614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7adda-f3c3-41a3-aafa-dcb83a1185ba None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:48.590082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e56ef7-c960-4cad-baf6-8a48bd5f0363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:48.590667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e56ef7-c960-4cad-baf6-8a48bd5f0363 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:48.590936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e56ef7-c960-4cad-baf6-8a48bd5f0363 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:48.591201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e56ef7-c960-4cad-baf6-8a48bd5f0363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:48.591907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e56ef7-c960-4cad-baf6-8a48bd5f0363 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=4cbb57adaf32457698920e6dabd42681, user.id=30dccb7e6b444fa294e3525c0876d5c4) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:48.592253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e56ef7-c960-4cad-baf6-8a48bd5f0363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:48.593664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e56ef7-c960-4cad-baf6-8a48bd5f0363 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:48.598784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 567/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:48 2022] GET /identity/v3/role_assignments?user.id=30dccb7e6b444fa294e3525c0876d5c4&scope.project.id=4cbb57adaf32457698920e6dabd42681&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:50.564520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81536a-e577-4d35-8b75-695d47d1f53d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:50.565036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81536a-e577-4d35-8b75-695d47d1f53d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:50.565130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81536a-e577-4d35-8b75-695d47d1f53d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:50.566668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 580/1147] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:50.571848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a5cc4-e65a-4553-80ac-b4fb33e0774d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:50.572288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a5cc4-e65a-4553-80ac-b4fb33e0774d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:50.572448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a5cc4-e65a-4553-80ac-b4fb33e0774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:50.590695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6f1a5cc4-e65a-4553-80ac-b4fb33e0774d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:50.591867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6f1a5cc4-e65a-4553-80ac-b4fb33e0774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:50.625425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:50.657769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 568/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:50.680459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81536a-e577-4d35-8b75-695d47d1f53d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:50.780775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81536a-e577-4d35-8b75-695d47d1f53d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:50.814187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:50.814810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:50.815087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:50.815357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:50.825419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:50.830072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:50.831118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:50.831972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 581/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:50.837323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a5cc4-e65a-4553-80ac-b4fb33e0774d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:50.935577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a5cc4-e65a-4553-80ac-b4fb33e0774d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:50.969014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:50.969631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:50.969914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:50.970187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:50.970876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:50.971175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:50.972574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:50.980462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 569/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:43:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:50.987014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:51.000169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:51.085168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676f102-f247-4c64-89b2-8548d86e8edf None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:51.118417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:51.119034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:51.119313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:51.119583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] PATH_INFO: `/v3/users/30dccb7e6b444fa294e3525c0876d5c4` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:51.120708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] RBAC: Authorizing `identity:get_user(user_id=30dccb7e6b444fa294e3525c0876d5c4)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:51.122119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:51.123882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 582/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:50 2022] GET /identity/v3/users/30dccb7e6b444fa294e3525c0876d5c4 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:51.129672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:51.163016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:51.230138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9c4c-7c68-4c64-a25c-c78c40a8e2c3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:51.265868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54e59e-5906-4a04-8d48-8f81730fa885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:51.266487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54e59e-5906-4a04-8d48-8f81730fa885 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:51.266766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54e59e-5906-4a04-8d48-8f81730fa885 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:51.267038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54e59e-5906-4a04-8d48-8f81730fa885 None admin] PATH_INFO: `/v3/projects/4cbb57adaf32457698920e6dabd42681` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:51.268173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54e59e-5906-4a04-8d48-8f81730fa885 None admin] RBAC: Authorizing `identity:get_project(project_id=4cbb57adaf32457698920e6dabd42681)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:51.269603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54e59e-5906-4a04-8d48-8f81730fa885 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:51.271224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 570/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:51 2022] GET /identity/v3/projects/4cbb57adaf32457698920e6dabd42681 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:51.277304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:51.353449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:51.376163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a700756-df62-4da4-8e9e-301298eecaa7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:51.409273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d49f1b-0154-4dcb-a4c9-8be5f5ee4d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:51.409937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d49f1b-0154-4dcb-a4c9-8be5f5ee4d3c None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:51.410216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d49f1b-0154-4dcb-a4c9-8be5f5ee4d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:51.410490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d49f1b-0154-4dcb-a4c9-8be5f5ee4d3c None admin] PATH_INFO: `/v3/projects/4cbb57adaf32457698920e6dabd42681/users/30dccb7e6b444fa294e3525c0876d5c4/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:51.412440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d49f1b-0154-4dcb-a4c9-8be5f5ee4d3c None admin] RBAC: Authorizing `identity:create_grant(project_id=4cbb57adaf32457698920e6dabd42681, user_id=30dccb7e6b444fa294e3525c0876d5c4, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:51.413864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d49f1b-0154-4dcb-a4c9-8be5f5ee4d3c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:51.427453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 583/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:43:51 2022] PUT /identity/v3/projects/4cbb57adaf32457698920e6dabd42681/users/30dccb7e6b444fa294e3525c0876d5c4/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:43:53.376767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e29c4-8172-4273-a2aa-10d22be346d8 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:53.377297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e29c4-8172-4273-a2aa-10d22be346d8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:53.377297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e29c4-8172-4273-a2aa-10d22be346d8 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:53.378654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 571/1154] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:53.383854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dd8be-be94-45af-b217-43654214df3f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:53.384257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dd8be-be94-45af-b217-43654214df3f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:53.384436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dd8be-be94-45af-b217-43654214df3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:53.402625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-621dd8be-be94-45af-b217-43654214df3f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:53.403822 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-621dd8be-be94-45af-b217-43654214df3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:53.475679 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 584/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:53.498859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e29c4-8172-4273-a2aa-10d22be346d8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:53.595375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:53.599799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e29c4-8172-4273-a2aa-10d22be346d8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:53.632622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:53.633178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:53.633456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:53.633728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:53.644529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:43:53.649175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:53.650196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:43:53.651051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 572/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:43:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:53.657115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dd8be-be94-45af-b217-43654214df3f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:53.720335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:53.753321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dd8be-be94-45af-b217-43654214df3f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:53.786568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:53.787161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:53.787439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:53.787705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:53.788382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:53.788681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:53.790021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:53.798353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 585/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:43:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:53.804215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:53.907010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc93068-148c-49ea-8e03-c0faaf77c22e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:53.940198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:53.940824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:53.941105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:53.941379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] PATH_INFO: `/v3/users/30dccb7e6b444fa294e3525c0876d5c4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:53.942751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] RBAC: Authorizing `identity:get_user(user_id=30dccb7e6b444fa294e3525c0876d5c4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:53.944201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:53.945781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 573/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:43:53 2022] GET /identity/v3/users/30dccb7e6b444fa294e3525c0876d5c4 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:53.951822 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:54.051901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70792e0f-11dd-48b7-a4cd-b2b332013c73 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:54.080782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:54.087429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:54.088051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:54.088364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:54.088642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:54.169677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:54.171623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:54.287735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 586/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:54.293751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:54.328791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:54.400375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d80e23-dea0-46d2-a84f-7718043e1a38 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:54.433724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed647ec0-e4e3-4047-8c5b-4f5c85755faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:54.434356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed647ec0-e4e3-4047-8c5b-4f5c85755faf None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:54.434633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed647ec0-e4e3-4047-8c5b-4f5c85755faf None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:54.434904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed647ec0-e4e3-4047-8c5b-4f5c85755faf None admin] PATH_INFO: `/v3/projects/4cbb57adaf32457698920e6dabd42681` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:54.436036 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed647ec0-e4e3-4047-8c5b-4f5c85755faf None admin] RBAC: Authorizing `identity:get_project(project_id=4cbb57adaf32457698920e6dabd42681)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:54.437490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed647ec0-e4e3-4047-8c5b-4f5c85755faf None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:54.439109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 574/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:43:54 2022] GET /identity/v3/projects/4cbb57adaf32457698920e6dabd42681 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:54.445499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:54.475378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:54.542017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6bb79-5325-4b44-a5c4-cc5bf63119b9 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:54.578045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041460af-b1b3-4ffe-bec7-eba8db7a401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:54.578655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041460af-b1b3-4ffe-bec7-eba8db7a401b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:54.578934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041460af-b1b3-4ffe-bec7-eba8db7a401b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:54.579206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041460af-b1b3-4ffe-bec7-eba8db7a401b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:54.579934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041460af-b1b3-4ffe-bec7-eba8db7a401b None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=4cbb57adaf32457698920e6dabd42681, user.id=30dccb7e6b444fa294e3525c0876d5c4) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:54.580273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041460af-b1b3-4ffe-bec7-eba8db7a401b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:54.581709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041460af-b1b3-4ffe-bec7-eba8db7a401b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:54.587503 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 587/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:43:54 2022] GET /identity/v3/role_assignments?user.id=30dccb7e6b444fa294e3525c0876d5c4&scope.project.id=4cbb57adaf32457698920e6dabd42681&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:56.629102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14e2f4-5102-435b-b7e9-838f7d0c7b37 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:56.629627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14e2f4-5102-435b-b7e9-838f7d0c7b37 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:56.629627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14e2f4-5102-435b-b7e9-838f7d0c7b37 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:56.630982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 575/1162] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:56.636306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd08cb-ef83-49b8-aa5c-f1c78b673e7a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:56.636727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd08cb-ef83-49b8-aa5c-f1c78b673e7a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:56.636858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd08cb-ef83-49b8-aa5c-f1c78b673e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:56.655076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-16cd08cb-ef83-49b8-aa5c-f1c78b673e7a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:56.656295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-16cd08cb-ef83-49b8-aa5c-f1c78b673e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:56.721093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 588/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:56.743094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14e2f4-5102-435b-b7e9-838f7d0c7b37 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:56.837551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:56.844826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14e2f4-5102-435b-b7e9-838f7d0c7b37 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:56.877607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:56.878200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:56.878475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:56.878743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:56.948802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:56.950400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:57.041275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:57.048941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 576/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:43:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:43:57.054884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd08cb-ef83-49b8-aa5c-f1c78b673e7a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:57.078682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:57.152326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd08cb-ef83-49b8-aa5c-f1c78b673e7a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:57.192352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8b58c-ce1b-4b22-940a-38c8601ccaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:57.193054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8b58c-ce1b-4b22-940a-38c8601ccaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:57.193362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8b58c-ce1b-4b22-940a-38c8601ccaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:57.193580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8b58c-ce1b-4b22-940a-38c8601ccaa3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:57.200326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8b58c-ce1b-4b22-940a-38c8601ccaa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:57.201922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8b58c-ce1b-4b22-940a-38c8601ccaa3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-1ce8b58c-ce1b-4b22-940a-38c8601ccaa3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:43:57.212932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:43:57.214101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:43:57.215118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 589/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:43:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:43:57.220536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:57.318566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230796-2cc6-4ef9-a093-a1d24d8e0d5d None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:57.352190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afe059-0659-40a6-81e0-afe7771a7bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:57.352797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afe059-0659-40a6-81e0-afe7771a7bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:57.353078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afe059-0659-40a6-81e0-afe7771a7bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:57.353352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afe059-0659-40a6-81e0-afe7771a7bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:57.353987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afe059-0659-40a6-81e0-afe7771a7bd7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:43:57.354289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afe059-0659-40a6-81e0-afe7771a7bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:57.355646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afe059-0659-40a6-81e0-afe7771a7bd7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:57.363247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 577/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 10 16:43:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:43:59.340488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88020e-07c1-43a9-a451-eb9605e053c6 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:59.341049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88020e-07c1-43a9-a451-eb9605e053c6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:59.341049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88020e-07c1-43a9-a451-eb9605e053c6 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:59.342871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 590/1167] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:43:59.348060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d23de-a638-4a44-aa2b-9c100a185fa6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:59.348496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d23de-a638-4a44-aa2b-9c100a185fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:59.348652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d23de-a638-4a44-aa2b-9c100a185fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:59.366803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b05d23de-a638-4a44-aa2b-9c100a185fa6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:43:59.367995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b05d23de-a638-4a44-aa2b-9c100a185fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:43:59.424079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:59.435669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 578/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:43:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:43:59.458369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88020e-07c1-43a9-a451-eb9605e053c6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:43:59.540642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:43:59.557537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88020e-07c1-43a9-a451-eb9605e053c6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:43:59.589743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c016-c0a6-45d5-938b-589d334c941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:43:59.590333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c016-c0a6-45d5-938b-589d334c941a None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:43:59.590655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c016-c0a6-45d5-938b-589d334c941a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:43:59.590874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c016-c0a6-45d5-938b-589d334c941a None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:43:59.591516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c016-c0a6-45d5-938b-589d334c941a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:43:59.593421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c016-c0a6-45d5-938b-589d334c941a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:43:59.608601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 591/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Wed Aug 10 16:43:59 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:44:01.650623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51d145-989e-4d3e-a6a3-ccc309022b21 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:01.651147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51d145-989e-4d3e-a6a3-ccc309022b21 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:01.651260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51d145-989e-4d3e-a6a3-ccc309022b21 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:01.652573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 579/1170] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:44:01.657878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a973c-d4a1-4e18-abfa-f1954a405efe None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:01.658239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a973c-d4a1-4e18-abfa-f1954a405efe None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:01.658432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a973c-d4a1-4e18-abfa-f1954a405efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:01.675971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-200a973c-d4a1-4e18-abfa-f1954a405efe None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:44:01.677188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-200a973c-d4a1-4e18-abfa-f1954a405efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:44:01.741310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 592/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:44:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:44:01.765646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51d145-989e-4d3e-a6a3-ccc309022b21 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:01.863859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51d145-989e-4d3e-a6a3-ccc309022b21 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:01.897982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc11752-81d7-47c9-93a2-1c084d0d0cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:01.898592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11752-81d7-47c9-93a2-1c084d0d0cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:01.898868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11752-81d7-47c9-93a2-1c084d0d0cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:01.899137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11752-81d7-47c9-93a2-1c084d0d0cb0 None admin] PATH_INFO: `/v3/domains/2ed46f07db824006bad5d19e7a4988a4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:01.909312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc11752-81d7-47c9-93a2-1c084d0d0cb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2ed46f07db824006bad5d19e7a4988a4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:01.910745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc11752-81d7-47c9-93a2-1c084d0d0cb0 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:01.912319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 580/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Wed Aug 10 16:44:01 2022] GET /identity/v3/domains/2ed46f07db824006bad5d19e7a4988a4 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:01.918058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a973c-d4a1-4e18-abfa-f1954a405efe None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:01.991870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:02.019170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a973c-d4a1-4e18-abfa-f1954a405efe None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:02.052263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c52e87-6cda-4eda-b467-c4e9cbfc5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:02.052865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c52e87-6cda-4eda-b467-c4e9cbfc5dcc None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:02.053145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c52e87-6cda-4eda-b467-c4e9cbfc5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:02.053413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c52e87-6cda-4eda-b467-c4e9cbfc5dcc None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:02.054042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c52e87-6cda-4eda-b467-c4e9cbfc5dcc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:02.055465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c52e87-6cda-4eda-b467-c4e9cbfc5dcc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:02.081158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 593/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:44:01 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:44:04.093792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5092a-40e9-4d91-bcda-ad33d7c756fd None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:04.094379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5092a-40e9-4d91-bcda-ad33d7c756fd None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:04.094379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5092a-40e9-4d91-bcda-ad33d7c756fd None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:04.095705 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 581/1174] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:44:04.100910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5a885-9061-48a6-bb22-4662d6169d7e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:04.101347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5a885-9061-48a6-bb22-4662d6169d7e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:04.101447 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5a885-9061-48a6-bb22-4662d6169d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:04.119528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ede5a885-9061-48a6-bb22-4662d6169d7e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:44:04.120717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ede5a885-9061-48a6-bb22-4662d6169d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:44:04.183488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 594/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:44:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:44:04.207497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5092a-40e9-4d91-bcda-ad33d7c756fd None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:04.257161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:04.306573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5092a-40e9-4d91-bcda-ad33d7c756fd None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:04.339005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c6454-fbc6-40b6-beea-e7e0ec40a1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:04.339624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6454-fbc6-40b6-beea-e7e0ec40a1ab None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:04.339902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6454-fbc6-40b6-beea-e7e0ec40a1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:04.340199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6454-fbc6-40b6-beea-e7e0ec40a1ab None admin] PATH_INFO: `/v3/domains/2ed46f07db824006bad5d19e7a4988a4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:04.341304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c6454-fbc6-40b6-beea-e7e0ec40a1ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=2ed46f07db824006bad5d19e7a4988a4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:04.343335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c6454-fbc6-40b6-beea-e7e0ec40a1ab None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:04.344974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 582/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Wed Aug 10 16:44:04 2022] GET /identity/v3/domains/2ed46f07db824006bad5d19e7a4988a4 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:04.350625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5a885-9061-48a6-bb22-4662d6169d7e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:04.403365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:04.447778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5a885-9061-48a6-bb22-4662d6169d7e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:04.481215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a26a5-f114-4c22-9955-2a2dab54cf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:04.481815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a26a5-f114-4c22-9955-2a2dab54cf41 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:04.482094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a26a5-f114-4c22-9955-2a2dab54cf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:04.482391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a26a5-f114-4c22-9955-2a2dab54cf41 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:04.483029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a26a5-f114-4c22-9955-2a2dab54cf41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:04.484603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a26a5-f114-4c22-9955-2a2dab54cf41 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:04.504480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 595/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:44:04 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:44:06.516449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874784de-e268-4fba-a8e5-47094145cdfc None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:06.516977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874784de-e268-4fba-a8e5-47094145cdfc None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:06.516977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874784de-e268-4fba-a8e5-47094145cdfc None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:06.518341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 583/1178] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:44:06.523343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed367dc-efb0-407b-a714-e90dea3c5cac None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:06.523742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed367dc-efb0-407b-a714-e90dea3c5cac None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:06.523909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed367dc-efb0-407b-a714-e90dea3c5cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:06.542212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-7ed367dc-efb0-407b-a714-e90dea3c5cac None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:44:06.543402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7ed367dc-efb0-407b-a714-e90dea3c5cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:44:06.607860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 596/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:44:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:44:06.630481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874784de-e268-4fba-a8e5-47094145cdfc None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:06.730915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874784de-e268-4fba-a8e5-47094145cdfc None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:06.747291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:06.764982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:06.765599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:06.765878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:06.766151 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:06.776369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:44:06.781121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:44:06.782275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:44:06.783353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 584/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:44:06.788303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed367dc-efb0-407b-a714-e90dea3c5cac None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:06.830682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:06.886283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed367dc-efb0-407b-a714-e90dea3c5cac None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:06.919140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:06.919789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:06.920073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:06.920382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:06.921041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:44:06.921348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:06.922715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:06.930569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 597/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:06.936867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:07.034365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1967-13f5-420f-be26-155b27ec3265 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:07.067166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:07.067784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:07.068066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:07.068371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] PATH_INFO: `/v3/users/2a2200df78a74a3fa4c05863fc1b1179` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:07.081641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] RBAC: Authorizing `identity:get_user(user_id=2a2200df78a74a3fa4c05863fc1b1179)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:07.083077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:07.084689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 585/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:44:06 2022] GET /identity/v3/users/2a2200df78a74a3fa4c05863fc1b1179 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:07.090433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:07.187515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3949a99-c4e4-4ec0-a4d7-2bbb8cf120cc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:07.189817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:07.220924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:07.221526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:07.221806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:07.222080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:07.289080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:07.290454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:07.385454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 598/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:44:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:44:07.391426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:07.425466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:07.492787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff4dc2-3cb1-4366-876f-cc13c93dc4bd None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:07.526060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a076b-b8e5-4103-b4eb-769ca3585df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:07.526678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a076b-b8e5-4103-b4eb-769ca3585df7 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:07.526956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a076b-b8e5-4103-b4eb-769ca3585df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:07.527227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a076b-b8e5-4103-b4eb-769ca3585df7 None admin] PATH_INFO: `/v3/projects/4030fc44ff964ce1959ecb15857887b3` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:07.528357 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a076b-b8e5-4103-b4eb-769ca3585df7 None admin] RBAC: Authorizing `identity:get_project(project_id=4030fc44ff964ce1959ecb15857887b3)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:07.530295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a076b-b8e5-4103-b4eb-769ca3585df7 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:07.531905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 586/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:44:07 2022] GET /identity/v3/projects/4030fc44ff964ce1959ecb15857887b3 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:07.538080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:07.541211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:07.634396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727a24b-3b47-4431-af10-5de3d6fe79cc None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:07.666881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54013d-d51b-4fc3-8721-660030b934e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:07.667473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54013d-d51b-4fc3-8721-660030b934e3 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:07.667753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54013d-d51b-4fc3-8721-660030b934e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:07.668024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54013d-d51b-4fc3-8721-660030b934e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:07.668762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54013d-d51b-4fc3-8721-660030b934e3 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=4030fc44ff964ce1959ecb15857887b3, user.id=2a2200df78a74a3fa4c05863fc1b1179) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:44:07.669068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54013d-d51b-4fc3-8721-660030b934e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:07.670471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54013d-d51b-4fc3-8721-660030b934e3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:07.675606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 599/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:44:07 2022] GET /identity/v3/role_assignments?user.id=2a2200df78a74a3fa4c05863fc1b1179&scope.project.id=4030fc44ff964ce1959ecb15857887b3&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:09.625758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213373-c1d9-4d64-97ed-3974468d7952 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:09.626317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213373-c1d9-4d64-97ed-3974468d7952 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:09.626416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213373-c1d9-4d64-97ed-3974468d7952 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:09.627729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 587/1186] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:44:09.632761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e71d-ee90-44a8-ba3b-1d6b04619bf3 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:09.633174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e71d-ee90-44a8-ba3b-1d6b04619bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:09.633317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e71d-ee90-44a8-ba3b-1d6b04619bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:09.650968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-33c3e71d-ee90-44a8-ba3b-1d6b04619bf3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:44:09.652134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-33c3e71d-ee90-44a8-ba3b-1d6b04619bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:44:09.736495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:09.742306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 600/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:44:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:44:09.767198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213373-c1d9-4d64-97ed-3974468d7952 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:09.866683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213373-c1d9-4d64-97ed-3974468d7952 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:09.869008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:09.900552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:09.901165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:09.901441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:09.901710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:09.911872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:44:09.916485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:44:09.917572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:44:09.919328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 588/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:44:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:44:09.924960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e71d-ee90-44a8-ba3b-1d6b04619bf3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:10.020368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e71d-ee90-44a8-ba3b-1d6b04619bf3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:10.054443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:10.055066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:10.055345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:10.055615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:10.056316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:44:10.056619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:10.058002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:10.068344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 601/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:10.074187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:10.181777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30699e5-7a41-4593-894e-eb61ad275e0b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:10.213930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:10.214539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:10.214818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:10.215086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] PATH_INFO: `/v3/users/2a2200df78a74a3fa4c05863fc1b1179` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:10.216206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] RBAC: Authorizing `identity:get_user(user_id=2a2200df78a74a3fa4c05863fc1b1179)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:10.218243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:10.219793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 589/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:44:10 2022] GET /identity/v3/users/2a2200df78a74a3fa4c05863fc1b1179 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:10.225572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:10.282177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:10.320672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91168735-eecb-4e2b-959b-05399ae0745c None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:10.353005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea5cde-9116-4e71-bc8f-782840559360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:10.353620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5cde-9116-4e71-bc8f-782840559360 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:10.353895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5cde-9116-4e71-bc8f-782840559360 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:10.354164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5cde-9116-4e71-bc8f-782840559360 None admin] PATH_INFO: `/v3/projects/4030fc44ff964ce1959ecb15857887b3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:10.355260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea5cde-9116-4e71-bc8f-782840559360 None admin] RBAC: Authorizing `identity:get_project(project_id=4030fc44ff964ce1959ecb15857887b3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:10.356716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea5cde-9116-4e71-bc8f-782840559360 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:10.358353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 602/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:44:10 2022] GET /identity/v3/projects/4030fc44ff964ce1959ecb15857887b3 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:10.364197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:10.394898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:10.466360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cab65-48c9-430e-b83f-ab9df09cb940 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:10.499278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411653bd-6218-41cf-9ac8-74ebd93b6bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:10.499927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411653bd-6218-41cf-9ac8-74ebd93b6bb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:10.500233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411653bd-6218-41cf-9ac8-74ebd93b6bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:10.500509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411653bd-6218-41cf-9ac8-74ebd93b6bb5 None admin] PATH_INFO: `/v3/projects/4030fc44ff964ce1959ecb15857887b3/users/2a2200df78a74a3fa4c05863fc1b1179/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:10.502391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411653bd-6218-41cf-9ac8-74ebd93b6bb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4030fc44ff964ce1959ecb15857887b3, user_id=2a2200df78a74a3fa4c05863fc1b1179, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:10.503816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411653bd-6218-41cf-9ac8-74ebd93b6bb5 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:10.516379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 590/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:44:10 2022] PUT /identity/v3/projects/4030fc44ff964ce1959ecb15857887b3/users/2a2200df78a74a3fa4c05863fc1b1179/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:44:12.505009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e00fb-b468-4996-a9d8-8610b5b98fb4 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:12.505541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e00fb-b468-4996-a9d8-8610b5b98fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:12.505633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e00fb-b468-4996-a9d8-8610b5b98fb4 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:12.506933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 603/1193] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:44:12.512212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027462a6-4be1-4ba5-9be2-ec5be2b530b8 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:12.512617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027462a6-4be1-4ba5-9be2-ec5be2b530b8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:12.512759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027462a6-4be1-4ba5-9be2-ec5be2b530b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:12.530338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-027462a6-4be1-4ba5-9be2-ec5be2b530b8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:44:12.531503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-027462a6-4be1-4ba5-9be2-ec5be2b530b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:44:12.595321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 591/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:44:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:44:12.617988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e00fb-b468-4996-a9d8-8610b5b98fb4 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:12.716389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e00fb-b468-4996-a9d8-8610b5b98fb4 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:12.741594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:12.749808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:12.750425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:12.750700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:12.750969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:12.760596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:44:12.765208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:44:12.766466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:44:12.767564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 604/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:44:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:44:12.772184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027462a6-4be1-4ba5-9be2-ec5be2b530b8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:12.796258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:12.871078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027462a6-4be1-4ba5-9be2-ec5be2b530b8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:12.904811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:12.905404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:12.905681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:12.905948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:12.906605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:44:12.906903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:12.908293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:12.916125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 592/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:44:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:12.922371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:13.020139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09375ce6-69b6-42b3-9e60-917a5ab06475 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:13.053564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:13.054182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:13.054459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:13.054730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] PATH_INFO: `/v3/users/2a2200df78a74a3fa4c05863fc1b1179` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:13.055835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] RBAC: Authorizing `identity:get_user(user_id=2a2200df78a74a3fa4c05863fc1b1179)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:13.057306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:13.058863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 605/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 10 16:44:12 2022] GET /identity/v3/users/2a2200df78a74a3fa4c05863fc1b1179 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:13.064619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:13.146877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:13.163214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123dbe5-faab-4a7e-b0f2-942b0ce30891 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:13.195928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:13.196552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:13.196832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:13.197105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:13.261668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:13.263102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:13.345227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:13.361735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 593/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:44:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:44:13.367762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:13.422461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:13.465860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee514-ba43-4b41-aa56-c14010931486 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:13.498843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483897db-a063-43e3-a77e-071a3cbeb49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:13.499464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483897db-a063-43e3-a77e-071a3cbeb49f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:13.499747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483897db-a063-43e3-a77e-071a3cbeb49f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:13.500019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483897db-a063-43e3-a77e-071a3cbeb49f None admin] PATH_INFO: `/v3/projects/4030fc44ff964ce1959ecb15857887b3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:13.501163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483897db-a063-43e3-a77e-071a3cbeb49f None admin] RBAC: Authorizing `identity:get_project(project_id=4030fc44ff964ce1959ecb15857887b3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:13.502585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483897db-a063-43e3-a77e-071a3cbeb49f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:13.504230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 606/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 10 16:44:13 2022] GET /identity/v3/projects/4030fc44ff964ce1959ecb15857887b3 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:13.510252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:44:13.604924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579959e-31d6-4d02-96b6-46884df7c1b2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:44:13.637242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49db2a6-f963-48c8-a96d-0eff6e6e5894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:44:13.637822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49db2a6-f963-48c8-a96d-0eff6e6e5894 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:44:13.638097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49db2a6-f963-48c8-a96d-0eff6e6e5894 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:44:13.638907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49db2a6-f963-48c8-a96d-0eff6e6e5894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:44:13.639613 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49db2a6-f963-48c8-a96d-0eff6e6e5894 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=4030fc44ff964ce1959ecb15857887b3, user.id=2a2200df78a74a3fa4c05863fc1b1179) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:44:13.639919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49db2a6-f963-48c8-a96d-0eff6e6e5894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:44:13.641355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49db2a6-f963-48c8-a96d-0eff6e6e5894 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:44:13.646806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 594/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:44:13 2022] GET /identity/v3/role_assignments?user.id=2a2200df78a74a3fa4c05863fc1b1179&scope.project.id=4030fc44ff964ce1959ecb15857887b3&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:44:23.494210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=38 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:44:23.644388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=76 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:09.114218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb5163-b968-4af2-929f-78d325375f9b None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:09.114810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb5163-b968-4af2-929f-78d325375f9b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:09.114911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb5163-b968-4af2-929f-78d325375f9b None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:09.116691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 607/1201] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:09.123652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839bdc9-48eb-4f3b-91f9-9c5ebc2c970c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:09.124123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839bdc9-48eb-4f3b-91f9-9c5ebc2c970c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:09.124238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839bdc9-48eb-4f3b-91f9-9c5ebc2c970c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:09.144877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c839bdc9-48eb-4f3b-91f9-9c5ebc2c970c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:09.146214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c839bdc9-48eb-4f3b-91f9-9c5ebc2c970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:09.226121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 595/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:09.253074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb5163-b968-4af2-929f-78d325375f9b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:09.384763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb5163-b968-4af2-929f-78d325375f9b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:09.424620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:09.425614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:09.425906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:09.426193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:09.440563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:09.442466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:09.465991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:09.467402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:09.468727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:09.468727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:09.468727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:09.468727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 608/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:09.473463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839bdc9-48eb-4f3b-91f9-9c5ebc2c970c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:09.598569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839bdc9-48eb-4f3b-91f9-9c5ebc2c970c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:09.634071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:09.642893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bb487-7139-43fb-8ddb-78f08e5312a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:09.643601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb487-7139-43fb-8ddb-78f08e5312a6 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:09.643952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb487-7139-43fb-8ddb-78f08e5312a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:09.644307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb487-7139-43fb-8ddb-78f08e5312a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:09.645278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bb487-7139-43fb-8ddb-78f08e5312a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:09.645445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bb487-7139-43fb-8ddb-78f08e5312a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:09.647105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bb487-7139-43fb-8ddb-78f08e5312a6 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:09.664196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 596/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:09.670544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:09.738344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:09.803254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5532d56-d2d6-4f1f-aba3-529a21988148 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:09.849773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97ac43-40d7-43a9-9c5a-2bda9c51a6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:09.850747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97ac43-40d7-43a9-9c5a-2bda9c51a6a3 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:09.851023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97ac43-40d7-43a9-9c5a-2bda9c51a6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:09.851321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97ac43-40d7-43a9-9c5a-2bda9c51a6a3 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:09.852057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97ac43-40d7-43a9-9c5a-2bda9c51a6a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:09.853694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97ac43-40d7-43a9-9c5a-2bda9c51a6a3 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:09.888176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 609/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 10 16:45:09 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:45:12.067954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b85ba-03df-4967-8a02-1300dadc4a56 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:12.068597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b85ba-03df-4967-8a02-1300dadc4a56 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:12.068597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b85ba-03df-4967-8a02-1300dadc4a56 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:12.071575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 597/1206] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:45:12.081096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779f78f-655f-45af-bed9-64a73d04b9b7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:12.081924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779f78f-655f-45af-bed9-64a73d04b9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:12.082202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779f78f-655f-45af-bed9-64a73d04b9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:12.107103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4779f78f-655f-45af-bed9-64a73d04b9b7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:12.108638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4779f78f-655f-45af-bed9-64a73d04b9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:12.189943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 610/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:12.213825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b85ba-03df-4967-8a02-1300dadc4a56 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:12.333905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b85ba-03df-4967-8a02-1300dadc4a56 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:12.377187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:12.378316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:12.378802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:12.379274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:12.396386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:45:12.404077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:12.405237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:12.406170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 598/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:12.412094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779f78f-655f-45af-bed9-64a73d04b9b7 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:12.463960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:12.533197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779f78f-655f-45af-bed9-64a73d04b9b7 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:12.574619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:12.575245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:12.575529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:12.575803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:12.576499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:12.576874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:12.578764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:12.592277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 611/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:45:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:12.593714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:12.646112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:12.719662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd61aa-ca50-4984-bd5c-819e5bec7e81 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:12.757366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:12.757992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:12.758269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:12.758538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:12.771451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:12.772964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:12.788951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:12.789906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:12.790790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:12.790790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:12.790790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:12.790790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 599/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:12.797544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:12.907963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:12.911484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f8db3-7acd-4e2c-9aeb-ac4e3c90c799 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:12.949518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:12.950190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:12.950455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:12.950746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:12.951479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:12.951795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:12.953251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:12.966251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 612/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:12.972409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:13.090454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b575586-a7cc-485d-993f-d769535e14cb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:13.092128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:13.135373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:13.136575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:13.136907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:13.137195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:13.146698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:13.148397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:45:13.162690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:13.163755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:13.164743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 600/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:13.169575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:13.287723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b8653-ec5c-4721-9e82-f5e1173a9bb2 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:13.324022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:13.324895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:13.325179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:13.325452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:13.326188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:13.326493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:13.329187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:13.352189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 613/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 10 16:45:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:45:13.355900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:13.493913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3648ad8-e10f-4ad1-abe0-a24b896e60ae None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:13.535424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:13.536032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:13.536335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:13.536628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:13.570043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:13.622485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:13.625232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:13.759214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 601/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:45:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:45:13.766358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:13.790174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:13.893783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6e5dc-fcd9-4d5c-8823-0c91aa70f9bf None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:13.939024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:13.940018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:13.940449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:13.940803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:13.952535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:13.954087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:13.971654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:13.972810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:13.973991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:13.973991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:13.973991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:13.973991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 614/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:13.980281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:14.054043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:14.111135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544e4d2-e0b2-436b-a3dd-729e778f2791 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:14.156599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:14.157511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:14.157795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:14.158064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:14.158711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:14.159008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:14.160397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:14.173125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 602/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:14.178894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:14.265969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:14.307039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8580069-2c38-483a-95e8-057c00165e02 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:14.346975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:14.347770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:14.348170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:14.350992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:14.359957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:14.361737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:45:14.376042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:14.377113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:14.377991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:14.380188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 615/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:45:14.386510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:14.508694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f28e2-c584-4955-9d58-af6ccb2993db None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:14.544571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:14.549081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5debc0-5dd8-411e-be5f-ad3ab4a2da15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:14.549945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5debc0-5dd8-411e-be5f-ad3ab4a2da15 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:14.550246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5debc0-5dd8-411e-be5f-ad3ab4a2da15 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:14.550609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5debc0-5dd8-411e-be5f-ad3ab4a2da15 None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:14.551444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5debc0-5dd8-411e-be5f-ad3ab4a2da15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:14.551991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5debc0-5dd8-411e-be5f-ad3ab4a2da15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:14.553576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5debc0-5dd8-411e-be5f-ad3ab4a2da15 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:14.571827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 603/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:45:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:14.573376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:14.695504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e773ca-a525-45b2-a7c3-00dc19dcccdd None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:14.735471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:14.743536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b31b95-d43f-493e-951a-108823865c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:14.744167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31b95-d43f-493e-951a-108823865c31 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:14.744456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31b95-d43f-493e-951a-108823865c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:14.744733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31b95-d43f-493e-951a-108823865c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:14.745483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b31b95-d43f-493e-951a-108823865c31 None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=836987c3b0a9483db61d8f31ad2af00b) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:14.745796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b31b95-d43f-493e-951a-108823865c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:14.747214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b31b95-d43f-493e-951a-108823865c31 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:14.764177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 616/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:45:14 2022] GET /identity/v3/role_assignments?user.id=836987c3b0a9483db61d8f31ad2af00b&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:16.872862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3443-cbb1-4371-a035-cbb821bcf85b None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:16.873418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3443-cbb1-4371-a035-cbb821bcf85b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:16.873637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3443-cbb1-4371-a035-cbb821bcf85b None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:16.875347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 604/1220] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:16.880854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ddf4-d9b5-48a6-9a32-f6e5d3cbc734 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:16.881269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ddf4-d9b5-48a6-9a32-f6e5d3cbc734 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:16.881401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ddf4-d9b5-48a6-9a32-f6e5d3cbc734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:16.901578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e6f6ddf4-d9b5-48a6-9a32-f6e5d3cbc734 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:16.903113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e6f6ddf4-d9b5-48a6-9a32-f6e5d3cbc734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:16.978467 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 617/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:17.000658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3443-cbb1-4371-a035-cbb821bcf85b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:17.110438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3443-cbb1-4371-a035-cbb821bcf85b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:17.147161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:17.147690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:17.147973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:17.148280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:17.161065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:45:17.167373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:17.168594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:17.169424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 605/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:45:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:17.174647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6ddf4-d9b5-48a6-9a32-f6e5d3cbc734 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:17.283730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6ddf4-d9b5-48a6-9a32-f6e5d3cbc734 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:17.308425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:17.320620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:17.321496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:17.321767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:17.322038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:17.322696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] RBAC: Adding query filter params (name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:17.323008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:17.324401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:17.339743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 618/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:17.341178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:17.389498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:17.451475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460aa5de-4541-4d1f-9897-4971349d54d3 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:17.489537 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:17.490177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:17.490461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:17.490737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:17.501339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:17.502781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:17.520514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:17.521521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:17.522472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:17.522472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:17.522472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:17.522472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 606/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:45:17.528726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:17.633385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adcaa5-4687-448a-9b5b-908cfc35a415 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:17.668332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:17.668872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:17.669148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:17.669413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:17.670039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:17.670336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:17.671693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:17.683521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 619/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:17.690136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:17.791391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:17.799510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a47a2-1d15-48ad-b05e-a813eecb024a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:17.837927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:17.838564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:17.838847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:17.839122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:17.846930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:17.848560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:45:17.860672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:17.861798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:17.862918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:17.868071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 607/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:17.868212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:17.916343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:17.973339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e23d2-eef6-4ea8-b566-548b5697affb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:18.012769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:18.013370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:18.013706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:18.013976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:18.014692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:18.014994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:18.016430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:18.034513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 620/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 10 16:45:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:18.040548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:18.148453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b07f5a-22ec-4cd8-ba8e-4028aef34b0f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:18.183220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:18.183851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:18.184171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:18.184547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:18.195646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:18.197366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:18.211658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:18.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:18.213938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:18.213938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:18.213938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:18.213938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 608/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:18.218589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:18.324869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995423d2-b454-4813-beef-1a1c88ce97cb None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:18.327727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:18.368456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:18.369106 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:18.369388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:18.369662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:18.370312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:18.370619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:18.371988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:18.386168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 621/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:45:18.392671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:18.402496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:18.510850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff145-91a0-4480-af78-9c7550c37302 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:18.552941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:18.553708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:18.554038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:18.554328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:18.561932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:18.563362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:45:18.577064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:18.578623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:18.579724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 609/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:45:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:45:18.584840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:18.702020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc6f0c-2c3d-49b7-8356-19bbb381683a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:18.746473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506169b9-1b2b-47b1-a379-2e52624d1f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:18.747277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506169b9-1b2b-47b1-a379-2e52624d1f80 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:18.747562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506169b9-1b2b-47b1-a379-2e52624d1f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:18.747836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506169b9-1b2b-47b1-a379-2e52624d1f80 None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:18.748578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506169b9-1b2b-47b1-a379-2e52624d1f80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:18.749134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506169b9-1b2b-47b1-a379-2e52624d1f80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:18.751531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506169b9-1b2b-47b1-a379-2e52624d1f80 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:18.765262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 622/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:45:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:18.772263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:18.854950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:18.900854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5cc7f-da0e-40ab-bf8b-45180b8dfec9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:18.942114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97686be6-8a52-4bfb-b6a8-86f848f6832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:18.943067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97686be6-8a52-4bfb-b6a8-86f848f6832d None admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:18.943452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97686be6-8a52-4bfb-b6a8-86f848f6832d None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:18.943764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97686be6-8a52-4bfb-b6a8-86f848f6832d None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/836987c3b0a9483db61d8f31ad2af00b/roles/6aaf096343b84cfea3eb6e8fd11ba645` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:18.961502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97686be6-8a52-4bfb-b6a8-86f848f6832d None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=836987c3b0a9483db61d8f31ad2af00b, role_id=6aaf096343b84cfea3eb6e8fd11ba645)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:18.962943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97686be6-8a52-4bfb-b6a8-86f848f6832d None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:18.977041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 610/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:45:18 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/836987c3b0a9483db61d8f31ad2af00b/roles/6aaf096343b84cfea3eb6e8fd11ba645 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:45:21.043612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416aec-94a3-47cb-ade3-01352cdd1566 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:21.044219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416aec-94a3-47cb-ade3-01352cdd1566 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:21.044219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416aec-94a3-47cb-ade3-01352cdd1566 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:21.045521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 623/1233] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:21.051809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934169c4-9a48-498e-b3ca-d155b1946b06 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:21.052251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934169c4-9a48-498e-b3ca-d155b1946b06 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:21.052419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934169c4-9a48-498e-b3ca-d155b1946b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:21.071881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-934169c4-9a48-498e-b3ca-d155b1946b06 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:21.073471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-934169c4-9a48-498e-b3ca-d155b1946b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:21.152928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 611/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:21.178265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416aec-94a3-47cb-ade3-01352cdd1566 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:21.203862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:21.286533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416aec-94a3-47cb-ade3-01352cdd1566 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:21.324482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:21.325040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:21.325318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:21.325586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:21.337407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:45:21.343264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:21.344868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:21.345715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 624/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 10 16:45:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:21.351673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934169c4-9a48-498e-b3ca-d155b1946b06 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:21.399597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:21.455132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934169c4-9a48-498e-b3ca-d155b1946b06 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:21.493360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:21.493967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:21.494243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:21.494509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:21.495160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:21.495456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:21.496851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:21.505345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 612/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 10 16:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:21.511059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:21.609782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:21.625803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d77cd3-781c-4614-aa05-4de49e0e513f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:21.662425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:21.663051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:21.663329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:21.663595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:21.673887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:21.675597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:21.691227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:21.692390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:21.693506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:21.693506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:21.693506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:21.693506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 625/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:21.698389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:21.810782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f268f-8461-410f-aeca-3057a4487a95 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:21.820090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:21.848234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:21.848878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:21.849161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:21.849442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:21.850085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:21.850495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:21.851987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:21.865592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 613/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:21.871670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:21.980393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ae9d9-984e-4fcc-b072-bab66042d29f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:22.025546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:22.026193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:22.026472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:22.026743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:22.031349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:22.037688 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:22.039658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:45:22.051992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:22.053580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:22.054513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:22.058581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 626/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:22.059630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:22.174425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aeb76-acdb-4d22-912a-bdf668210a5f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:22.213364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:22.214082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:22.214372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:22.214766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:22.215515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:22.215947 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:22.217550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:22.235996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 614/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 10 16:45:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:45:22.241814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:22.350613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8a9d-bd0d-485b-8ff5-fee11f301446 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:22.385190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:22.385802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:22.386079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:22.386352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:22.458143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:22.466570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:22.468211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:22.572520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 627/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:45:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:45:22.578662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:22.607534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:22.687738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86588f0e-9807-418a-9e5c-14ac6892c42b None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:22.727145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:22.727703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:22.727979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:22.728283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:22.739969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:22.741459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:22.756468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:22.758005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:22.759485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:22.759485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:22.759485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:22.759485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 615/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:22.763298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:22.863996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:22.870009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5cd6a-adcd-415b-9d6d-f1cffbea326d None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:22.910708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:22.911338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:22.911619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:22.911900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:22.912575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:22.912918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:22.914331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:22.926399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 628/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:22.932303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:23.029263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:23.058479 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43554b77-d786-4466-a111-7a59b06a37a2 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:23.099933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:23.100620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:23.100840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:23.101108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:23.107634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:23.109112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:45:23.121216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:23.122619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:23.123488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 616/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:23.128502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:23.228385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08854d-54c6-42c1-8c7e-b8bc67810067 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:23.263806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51808264-ae68-4a17-9f98-be4ddb2d6a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:23.264420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51808264-ae68-4a17-9f98-be4ddb2d6a3b None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:23.264695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51808264-ae68-4a17-9f98-be4ddb2d6a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:23.264969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51808264-ae68-4a17-9f98-be4ddb2d6a3b None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:23.265672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51808264-ae68-4a17-9f98-be4ddb2d6a3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:23.266491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51808264-ae68-4a17-9f98-be4ddb2d6a3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:23.267974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51808264-ae68-4a17-9f98-be4ddb2d6a3b None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:23.285518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 629/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:23.286841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:23.393605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15587ae9-c1b2-4f57-80b2-0cb9d2ffec55 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:23.434930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda73327-6d00-44bb-8c30-e2e9b357c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:23.435655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda73327-6d00-44bb-8c30-e2e9b357c280 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:23.436004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda73327-6d00-44bb-8c30-e2e9b357c280 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:23.436320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda73327-6d00-44bb-8c30-e2e9b357c280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:23.437048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda73327-6d00-44bb-8c30-e2e9b357c280 None admin] RBAC: Adding query filter params (role.id=6aaf096343b84cfea3eb6e8fd11ba645, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=836987c3b0a9483db61d8f31ad2af00b) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:23.437453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda73327-6d00-44bb-8c30-e2e9b357c280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:23.439637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda73327-6d00-44bb-8c30-e2e9b357c280 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:23.447031 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 617/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:45:23 2022] GET /identity/v3/role_assignments?user.id=836987c3b0a9483db61d8f31ad2af00b&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=6aaf096343b84cfea3eb6e8fd11ba645 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:25.530023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa78b4-7fa2-442b-86ea-484536e87b43 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:25.530722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa78b4-7fa2-442b-86ea-484536e87b43 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:25.530722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa78b4-7fa2-442b-86ea-484536e87b43 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:25.534886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 630/1247] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:25.537781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b0f3-7fd7-4362-b25c-fc11fad51cf6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:25.538271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b0f3-7fd7-4362-b25c-fc11fad51cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:25.538365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b0f3-7fd7-4362-b25c-fc11fad51cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:25.544795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:25.558312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-61e4b0f3-7fd7-4362-b25c-fc11fad51cf6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:25.559596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-61e4b0f3-7fd7-4362-b25c-fc11fad51cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:25.636873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 618/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:25.665297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa78b4-7fa2-442b-86ea-484536e87b43 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:25.687033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:25.782646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa78b4-7fa2-442b-86ea-484536e87b43 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:25.823933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:25.824662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:25.824940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:25.825250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:25.839899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:45:25.845578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:25.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:25.847691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 631/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:45:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:25.852826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b0f3-7fd7-4362-b25c-fc11fad51cf6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:25.964972 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b0f3-7fd7-4362-b25c-fc11fad51cf6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:26.005143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:26.011099 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:26.011802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:26.012059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:26.012395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:26.013055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:26.013398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:26.014830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:26.025077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 619/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:45:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:26.031630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:26.124716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:26.153401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1e050-4454-49fe-88cb-424ad983af3a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:26.192652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:26.193318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:26.193610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:26.193875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:26.206392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:26.208043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:26.225073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:26.230834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:26.231945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:26.231945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:26.231945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:26.231945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 632/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:26.232248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:26.347596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371535-c243-4780-b2e5-ecf436cfd631 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:26.387147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:26.387763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:26.388041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:26.388406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:26.389043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:26.389344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:26.390705 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:26.404470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 620/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:45:26.410863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:26.528351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156b42-5a82-47c6-978c-1d4a37897fbd None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:26.558595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:26.567906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:26.568573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:26.568896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:26.569172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:26.579032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:26.580647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:45:26.592129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:26.593205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:26.594692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 633/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:45:26.599801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:26.668072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:26.721314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f833a-370e-4c85-83da-1059411ffb2c None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:26.757821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:26.758427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:26.758705 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:26.758975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:26.759688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:26.759990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:26.761447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:26.783584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 621/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 10 16:45:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:45:26.789335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:26.904849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967d5e5-9bd8-4c42-afec-3cbd89ec2d24 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:26.942411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:26.943068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:26.943361 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:26.943680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:27.018408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:27.032845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:27.034432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:27.151629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 634/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:45:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:45:27.157711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:27.283833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2fd53-db90-4786-a552-a46f28722a22 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:27.298224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:27.322943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:27.323572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:27.323863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:27.324115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:27.335986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:27.337562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:27.353189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:27.354470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:27.355561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:27.355561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:27.355561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:27.355561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 622/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:27.360584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:27.460743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:27.474719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb010d-331f-4813-8e82-bb560d636512 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:27.513555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:27.514202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:27.514449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:27.514764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:27.515417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:27.515713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:27.517224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:27.534234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 635/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:45:27.536492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:27.654746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e03e8b-208e-45c0-ae69-5b00f77166ec None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:27.690180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:27.690815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:27.691093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:27.691421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:27.699186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:27.700868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:45:27.712564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:27.713940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:27.715181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 623/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:45:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:27.720243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:27.837194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847db91-cb96-42f7-8353-6d4318e91069 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:27.876055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84117a18-1da9-46a9-9c01-0d0d0d8b7a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:27.876803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117a18-1da9-46a9-9c01-0d0d0d8b7a8e None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:27.877189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117a18-1da9-46a9-9c01-0d0d0d8b7a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:27.877513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117a18-1da9-46a9-9c01-0d0d0d8b7a8e None admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:27.878240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84117a18-1da9-46a9-9c01-0d0d0d8b7a8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:27.878597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84117a18-1da9-46a9-9c01-0d0d0d8b7a8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:27.880401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84117a18-1da9-46a9-9c01-0d0d0d8b7a8e None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:27.893450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 636/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:45:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:27.899727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:27.929408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:28.019413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620cb0-37be-42cd-9719-a60d3f6d53ce None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:28.064987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae56cfb-e0a1-4346-bf01-2b92bae706e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:28.065669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae56cfb-e0a1-4346-bf01-2b92bae706e4 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:28.065908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae56cfb-e0a1-4346-bf01-2b92bae706e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:28.066158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae56cfb-e0a1-4346-bf01-2b92bae706e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:28.066886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae56cfb-e0a1-4346-bf01-2b92bae706e4 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=836987c3b0a9483db61d8f31ad2af00b) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:28.067197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae56cfb-e0a1-4346-bf01-2b92bae706e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:28.068679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae56cfb-e0a1-4346-bf01-2b92bae706e4 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:28.074736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 624/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:45:27 2022] GET /identity/v3/role_assignments?user.id=836987c3b0a9483db61d8f31ad2af00b&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:30.153890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0e59-dc65-48d1-b720-7cf5d80c339c None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:30.154416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0e59-dc65-48d1-b720-7cf5d80c339c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:30.154416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0e59-dc65-48d1-b720-7cf5d80c339c None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:30.155761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 637/1261] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:30.160994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfaa15-6dac-4cb0-9a55-ad320f7af9a2 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:30.161367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfaa15-6dac-4cb0-9a55-ad320f7af9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:30.161542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfaa15-6dac-4cb0-9a55-ad320f7af9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:30.179187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-dcdfaa15-6dac-4cb0-9a55-ad320f7af9a2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:30.180390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-dcdfaa15-6dac-4cb0-9a55-ad320f7af9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:30.245086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 625/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:30.267835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0e59-dc65-48d1-b720-7cf5d80c339c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:30.278432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:30.367219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0e59-dc65-48d1-b720-7cf5d80c339c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:30.400715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:30.401339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:30.401622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:30.401897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:30.411996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:45:30.416696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:30.417742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:30.419928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 638/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:30.426423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfaa15-6dac-4cb0-9a55-ad320f7af9a2 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:30.444985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:30.523418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfaa15-6dac-4cb0-9a55-ad320f7af9a2 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:30.557027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:30.557622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:30.557897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:30.558170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:30.558817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:30.559113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:30.560487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:30.568970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 626/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:45:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:30.575176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:30.639876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:30.671666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a1d9e-1499-489d-a70c-2f9c73912b27 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:30.705252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:30.705867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:30.706148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:30.706420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:30.716027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:30.717490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:30.731419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:30.736226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:30.737348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:30.737348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:30.737348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:30.737348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 639/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:30.738802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:30.817484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:30.836410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca64360-ce96-446d-b711-05e42bb9e84e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:30.869268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:30.869858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:30.870135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:30.870408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:30.871040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:30.871340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:30.872729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:30.883989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 627/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:30.889492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:30.988344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6ce0-f8ce-473b-9947-dd1ec10b3349 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:31.011187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:31.026762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:31.027393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:31.027674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:31.027948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:31.035945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:31.037416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:45:31.048772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:31.049820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:31.050894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 640/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:31.055618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:31.153125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b32078-c0e6-4db7-b360-1ebb3dc8ce23 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:31.186415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:31.186992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:31.187271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:31.187540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:31.188283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:31.188590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:31.190004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:31.198633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:31.207381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 628/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 10 16:45:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:31.213138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:31.309607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3c0b9-7ce4-4219-92f3-f72025f439f7 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:31.341603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:31.342216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:31.342491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:31.342764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:31.353357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:31.354783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:31.368300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:31.369398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:31.370610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:31.370610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:31.370610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:31.370610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 641/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:31.374931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:31.473547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98d002-bcdd-49d2-9db4-2df1049398bb None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:31.505908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:31.506510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:31.506786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:31.507053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:31.507678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:31.507976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:31.509352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:31.520623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 629/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:31.526226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:31.557454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:31.624826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c386-5237-46d3-85f9-114bbb5e86a8 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:31.659248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:31.659864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:31.660195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:31.660471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:31.666998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:31.668461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:45:31.679588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:31.680600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:31.681912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 642/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:31.686568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:31.744054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:31.785057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9e648-dbd4-4ed7-95c8-9add9e8a80c0 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:31.817150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9a0e6-c585-484e-84dd-c845b9d7ae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:31.817732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a0e6-c585-484e-84dd-c845b9d7ae0c None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:31.818005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a0e6-c585-484e-84dd-c845b9d7ae0c None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:31.818273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a0e6-c585-484e-84dd-c845b9d7ae0c None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:31.818944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9a0e6-c585-484e-84dd-c845b9d7ae0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:31.819241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9a0e6-c585-484e-84dd-c845b9d7ae0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:31.820682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9a0e6-c585-484e-84dd-c845b9d7ae0c None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:31.832935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 630/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:31.838888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:31.929538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:31.939440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f82c39-ca6b-42cd-ac14-71c0db091012 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:31.975158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ee8b-2825-446b-b934-aac33ef77121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:31.975860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ee8b-2825-446b-b934-aac33ef77121 None admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:31.976193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ee8b-2825-446b-b934-aac33ef77121 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:31.976508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ee8b-2825-446b-b934-aac33ef77121 None admin] PATH_INFO: `/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/836987c3b0a9483db61d8f31ad2af00b/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:31.978985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0ee8b-2825-446b-b934-aac33ef77121 None admin] RBAC: Authorizing `identity:create_grant(project_id=384fce8c44ca4ff4be15fc9495992f7c, user_id=836987c3b0a9483db61d8f31ad2af00b, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:31.980846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0ee8b-2825-446b-b934-aac33ef77121 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:31.997243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 643/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 10 16:45:31 2022] PUT /identity/v3/projects/384fce8c44ca4ff4be15fc9495992f7c/users/836987c3b0a9483db61d8f31ad2af00b/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:45:33.993788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3709ec7-d997-4a01-928e-183e8c6ba8cf None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:33.994344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3709ec7-d997-4a01-928e-183e8c6ba8cf None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:33.994344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3709ec7-d997-4a01-928e-183e8c6ba8cf None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:33.995736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 631/1274] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:34.001771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d16f-f1f8-468f-b269-c9e125bfbe3c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:34.002186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d16f-f1f8-468f-b269-c9e125bfbe3c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:34.002476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d16f-f1f8-468f-b269-c9e125bfbe3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:34.023985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9414d16f-f1f8-468f-b269-c9e125bfbe3c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:34.025387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9414d16f-f1f8-468f-b269-c9e125bfbe3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:34.090555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 644/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:34.114600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3709ec7-d997-4a01-928e-183e8c6ba8cf None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:34.213551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3709ec7-d997-4a01-928e-183e8c6ba8cf None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:34.230502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:34.247130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:34.247689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:34.247963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:34.248266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:34.258016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:45:34.262709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:34.263764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:34.264734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 632/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 10 16:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:34.270295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d16f-f1f8-468f-b269-c9e125bfbe3c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:34.348113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:34.366271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d16f-f1f8-468f-b269-c9e125bfbe3c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:34.401272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:34.401879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:34.402162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:34.402432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:34.403088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:34.403392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:34.404804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:34.415823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 645/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 10 16:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:34.421654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:34.524310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aec92-f19c-4c12-944d-497096bfe420 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:34.556585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:34.557204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:34.557485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:34.557760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:34.568484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:34.569926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:34.584435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:34.585426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:34.586469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:34.586469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:34.586469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:34.586469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 633/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:34.591324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:34.687723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6128-9625-496f-8dae-c7feb401643f None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:34.721265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:34.721869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:34.722151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:34.722425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:34.723058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:34.723360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:34.724744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:34.736688 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 646/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:34.742421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:34.774566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:34.843017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a150-bf97-4a6e-9b89-447811fe630f None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:34.874818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:34.875915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:34.876214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:34.876481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:34.883562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:34.885013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:45:34.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:34.897456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:34.898404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 634/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:34.903378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:34.907128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:35.009141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18a9e6-f13c-4c9c-b108-5005c7e609d1 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:35.043207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:35.043736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:35.044011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:35.044316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:35.045021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:35.045318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:35.046890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:35.064114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 647/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 10 16:45:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:35.069832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:35.157826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:35.167666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918f5f0-d08d-4271-ba18-78d794c6962e None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:35.201607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:35.202243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:35.202527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:35.202801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:35.267993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:35.269584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:35.359627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:35.370196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 635/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 10 16:45:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:45:35.376084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:35.435957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:35.473814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6cc4-a2fb-4455-b224-4c0a3f92e0e4 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:35.506868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:35.507491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:35.507774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:35.508047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:35.517851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:35.519287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:45:35.533802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:35.534871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:45:35.536032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:45:35.536032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:45:35.536032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:35.536032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 648/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:35.540783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:35.637282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51594c7e-58f1-4e54-ab39-7f50efadbc49 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:35.670526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:35.671123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:35.671398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:35.671670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:35.672354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:35.672655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:35.674011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:35.685056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 636/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 10 16:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:35.690549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:35.789874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad510a-cac8-4b67-832f-717dd9547a2a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:35.803212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:35.824562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:35.825182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:35.825459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:35.825726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:35.832672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:35.834114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:45:35.845360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:35.846414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: ERROR keystone.server.flask.application Aug 10 16:45:35.847508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 649/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 10 16:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:35.852420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:35.903945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:35.953459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc51f-26a7-4654-a009-17f3676a5ea9 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:35.990218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35db957-68d1-4193-87f0-b9a8f3348aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:35.991020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35db957-68d1-4193-87f0-b9a8f3348aea None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:35.991420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35db957-68d1-4193-87f0-b9a8f3348aea None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:35.991839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35db957-68d1-4193-87f0-b9a8f3348aea None admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:35.992820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35db957-68d1-4193-87f0-b9a8f3348aea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:35.993257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35db957-68d1-4193-87f0-b9a8f3348aea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:35.995019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35db957-68d1-4193-87f0-b9a8f3348aea None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:36.010379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 637/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 10 16:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:36.017544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:36.126935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e03e-01be-4bfa-9d9b-500c184d2f8a None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:36.173831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71f5c4-0708-4a21-b201-2fe9bfd9ebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:36.174522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f5c4-0708-4a21-b201-2fe9bfd9ebe8 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:36.174855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f5c4-0708-4a21-b201-2fe9bfd9ebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:36.175127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f5c4-0708-4a21-b201-2fe9bfd9ebe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:36.176231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71f5c4-0708-4a21-b201-2fe9bfd9ebe8 None admin] RBAC: Adding query filter params (role.id=d175b37bcc59450f897dbc44ba9ac182, scope.project.id=384fce8c44ca4ff4be15fc9495992f7c, user.id=836987c3b0a9483db61d8f31ad2af00b) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:36.176701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71f5c4-0708-4a21-b201-2fe9bfd9ebe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:36.178960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71f5c4-0708-4a21-b201-2fe9bfd9ebe8 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:36.187171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 650/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 10 16:45:36 2022] GET /identity/v3/role_assignments?user.id=836987c3b0a9483db61d8f31ad2af00b&scope.project.id=384fce8c44ca4ff4be15fc9495992f7c&role.id=d175b37bcc59450f897dbc44ba9ac182 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:38.191557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b9c3a-2b1a-47f9-a370-c05f0faf5907 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:38.192087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b9c3a-2b1a-47f9-a370-c05f0faf5907 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:38.192087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b9c3a-2b1a-47f9-a370-c05f0faf5907 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:38.193491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 638/1288] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:38.198844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19587aa-db7e-4fee-b27e-fdb9b36c8ee2 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:38.199258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19587aa-db7e-4fee-b27e-fdb9b36c8ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:38.199389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19587aa-db7e-4fee-b27e-fdb9b36c8ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:38.216999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d19587aa-db7e-4fee-b27e-fdb9b36c8ee2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:38.218230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d19587aa-db7e-4fee-b27e-fdb9b36c8ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:38.247021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:38.284541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 651/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:38.306772 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b9c3a-2b1a-47f9-a370-c05f0faf5907 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:38.403596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b9c3a-2b1a-47f9-a370-c05f0faf5907 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:38.414466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:38.438819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:38.439440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:38.439716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:38.439983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:38.440587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:38.441988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:38.450596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:38.451753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:38.452658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 639/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:45:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:38.457689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19587aa-db7e-4fee-b27e-fdb9b36c8ee2 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:38.559574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19587aa-db7e-4fee-b27e-fdb9b36c8ee2 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:38.591484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f308294-f69c-49f7-ae12-2f1e0a7a9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:38.592116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f308294-f69c-49f7-ae12-2f1e0a7a9486 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:38.592518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f308294-f69c-49f7-ae12-2f1e0a7a9486 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:38.592692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f308294-f69c-49f7-ae12-2f1e0a7a9486 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:38.593307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f308294-f69c-49f7-ae12-2f1e0a7a9486 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:38.593603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f308294-f69c-49f7-ae12-2f1e0a7a9486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:38.595472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f308294-f69c-49f7-ae12-2f1e0a7a9486 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:38.600084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 652/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 10 16:45:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:45:38.605927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:38.704782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefba5c-b51e-4566-9499-501f52c94819 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:38.737196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263848c-fadb-43c8-8af3-ed162d0a46b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:38.737780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263848c-fadb-43c8-8af3-ed162d0a46b1 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:38.738055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263848c-fadb-43c8-8af3-ed162d0a46b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:38.738325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263848c-fadb-43c8-8af3-ed162d0a46b1 None admin] PATH_INFO: `/v3/services` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:38.738938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263848c-fadb-43c8-8af3-ed162d0a46b1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:38.739233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263848c-fadb-43c8-8af3-ed162d0a46b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:38.740618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263848c-fadb-43c8-8af3-ed162d0a46b1 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:38.744857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 640/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 10 16:45:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:40.720445 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762270-d7d8-4464-8c3e-7b72158ac682 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:40.720961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762270-d7d8-4464-8c3e-7b72158ac682 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:40.721057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762270-d7d8-4464-8c3e-7b72158ac682 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:40.722448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 653/1293] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:40.727606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee2453-d59e-40e1-9963-fa4021d9cc30 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:40.727998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee2453-d59e-40e1-9963-fa4021d9cc30 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:40.728176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee2453-d59e-40e1-9963-fa4021d9cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:40.745924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f0ee2453-d59e-40e1-9963-fa4021d9cc30 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:40.747117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f0ee2453-d59e-40e1-9963-fa4021d9cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:40.776645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:40.816891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 641/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:40.841418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e762270-d7d8-4464-8c3e-7b72158ac682 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:40.871560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:40.940253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e762270-d7d8-4464-8c3e-7b72158ac682 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:40.972527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e26ff-34fe-4b79-8663-b4fcb43ea4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:40.973062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e26ff-34fe-4b79-8663-b4fcb43ea4f0 None admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:40.973340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e26ff-34fe-4b79-8663-b4fcb43ea4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:40.973607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e26ff-34fe-4b79-8663-b4fcb43ea4f0 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:40.974244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e26ff-34fe-4b79-8663-b4fcb43ea4f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:40.975845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e26ff-34fe-4b79-8663-b4fcb43ea4f0 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:40.994401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 654/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 10 16:45:40 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:45:43.019047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0961-32ad-46e2-acdb-4b57c4da22de None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:43.019601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0961-32ad-46e2-acdb-4b57c4da22de None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:43.019601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0961-32ad-46e2-acdb-4b57c4da22de None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:43.020988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 642/1296] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:43.025911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419513c2-d595-4b6e-ac10-b6e971377ddf None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:43.026346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419513c2-d595-4b6e-ac10-b6e971377ddf None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:43.026462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419513c2-d595-4b6e-ac10-b6e971377ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:43.044454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-419513c2-d595-4b6e-ac10-b6e971377ddf None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:43.045651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-419513c2-d595-4b6e-ac10-b6e971377ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:43.116719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 655/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:43.145359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e0961-32ad-46e2-acdb-4b57c4da22de None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:43.245067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e0961-32ad-46e2-acdb-4b57c4da22de None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:43.277395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:43.277953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:43.278226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:43.278492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:43.279061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:43.280442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:43.289364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:43.290536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:43.291423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 643/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:45:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:43.296780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419513c2-d595-4b6e-ac10-b6e971377ddf None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:43.314412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:43.393283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419513c2-d595-4b6e-ac10-b6e971377ddf None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:43.426272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:43.426874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:43.427152 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:43.427422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:43.428042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:43.428374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:43.429758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:43.434924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 656/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 10 16:45:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:43.450323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:43.485515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:43.553309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4174f5a-f7a6-458a-ae53-5fdc9dfd2618 None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:43.587014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80674a46-88d2-4b8a-a4ba-012379fc9f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:43.587615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80674a46-88d2-4b8a-a4ba-012379fc9f34 None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:43.587890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80674a46-88d2-4b8a-a4ba-012379fc9f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:43.588168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80674a46-88d2-4b8a-a4ba-012379fc9f34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:43.588884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80674a46-88d2-4b8a-a4ba-012379fc9f34 None admin] RBAC: Adding query filter params (interface=public, service_id=3246b7ebaf6142d38c74ab4a200a0d11, region_id=RegionOne) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:43.589192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80674a46-88d2-4b8a-a4ba-012379fc9f34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:43.590551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80674a46-88d2-4b8a-a4ba-012379fc9f34 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:43.595436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 644/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 10 16:45:43 2022] GET /identity/v3/endpoints?service_id=3246b7ebaf6142d38c74ab4a200a0d11&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:43.601088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:43.681548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:43.700454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12052-c60b-42fe-88e1-b31ae9554552 None admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:43.733072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabacb1a-d47d-49c9-98d6-728f96241a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:43.733674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabacb1a-d47d-49c9-98d6-728f96241a04 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:43.733951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabacb1a-d47d-49c9-98d6-728f96241a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:43.734226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabacb1a-d47d-49c9-98d6-728f96241a04 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:43.734820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabacb1a-d47d-49c9-98d6-728f96241a04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:43.736236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabacb1a-d47d-49c9-98d6-728f96241a04 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:43.742849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 657/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 10 16:45:43 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:45:45.722466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68d8c-9a72-4f60-9ce9-df944bec5896 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:45.722994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68d8c-9a72-4f60-9ce9-df944bec5896 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:45.722994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68d8c-9a72-4f60-9ce9-df944bec5896 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:45.724383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 645/1302] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:45:45.729481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a649ab-1fdc-4fc5-819d-d592a8ac8ea9 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:45.729869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a649ab-1fdc-4fc5-819d-d592a8ac8ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:45.730033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a649ab-1fdc-4fc5-819d-d592a8ac8ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:45.747755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-81a649ab-1fdc-4fc5-819d-d592a8ac8ea9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:45:45.748951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-81a649ab-1fdc-4fc5-819d-d592a8ac8ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:45:45.820858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 658/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:45:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:45:45.843854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed68d8c-9a72-4f60-9ce9-df944bec5896 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:45.937648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:45.945285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed68d8c-9a72-4f60-9ce9-df944bec5896 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:45.978285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:45.978895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:45.979246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:45.979586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:45.980228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:45.981915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:45:45.992651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:45:45.993855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application with Lock( Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: ERROR keystone.server.flask.application Aug 10 16:45:45.994697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 646/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 10 16:45:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:45:46.001452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a649ab-1fdc-4fc5-819d-d592a8ac8ea9 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:46.075382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:46.098786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a649ab-1fdc-4fc5-819d-d592a8ac8ea9 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:46.131341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e627d8d-67d9-435b-8140-788701237811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:46.131941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e627d8d-67d9-435b-8140-788701237811 None admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:46.132246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e627d8d-67d9-435b-8140-788701237811 None admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:46.132520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e627d8d-67d9-435b-8140-788701237811 None admin] PATH_INFO: `/v3/services` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:46.133142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e627d8d-67d9-435b-8140-788701237811 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:45:46.133447 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e627d8d-67d9-435b-8140-788701237811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:46.134794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e627d8d-67d9-435b-8140-788701237811 None admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:46.139725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 659/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 10 16:45:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:45:46.146047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:45:46.243931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab10fe-c05d-4b47-8c4d-d8886951687a None admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:45:46.276651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da532f44-7ba3-450c-b2d0-b6ae065a3bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:45:46.277248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532f44-7ba3-450c-b2d0-b6ae065a3bf3 None admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:45:46.277526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532f44-7ba3-450c-b2d0-b6ae065a3bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:45:46.277794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532f44-7ba3-450c-b2d0-b6ae065a3bf3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:45:46.278448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da532f44-7ba3-450c-b2d0-b6ae065a3bf3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:45:46.279806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da532f44-7ba3-450c-b2d0-b6ae065a3bf3 None admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:45:46.298735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 647/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:45:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:45:56.137023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=30 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:45:56.294491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:46:51.879973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c3793-ff2f-4a4a-847c-36dccaca13be None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:51.880561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c3793-ff2f-4a4a-847c-36dccaca13be None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:51.880561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c3793-ff2f-4a4a-847c-36dccaca13be None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:51.882249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 660/1307] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:46:51.887429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78a62-a8d9-4c3f-aa0c-0df1be54a7ca None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:51.887920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78a62-a8d9-4c3f-aa0c-0df1be54a7ca None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:51.888001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78a62-a8d9-4c3f-aa0c-0df1be54a7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:51.991587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-87d78a62-a8d9-4c3f-aa0c-0df1be54a7ca None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:46:51.992893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-87d78a62-a8d9-4c3f-aa0c-0df1be54a7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:52.118466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 648/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:46:52.169664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604fcfd-4b87-41c2-b4b0-7e77ea20f54d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:52.169876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604fcfd-4b87-41c2-b4b0-7e77ea20f54d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:52.170120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604fcfd-4b87-41c2-b4b0-7e77ea20f54d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:52.171790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 661/1309] 10.222.0.25 () {54 vars in 965 bytes} [Wed Aug 10 16:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:46:52.178487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904874ee-1a7a-4228-ae32-dd2aa815eecf None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:52.178773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904874ee-1a7a-4228-ae32-dd2aa815eecf None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:52.179129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904874ee-1a7a-4228-ae32-dd2aa815eecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:52.278553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-904874ee-1a7a-4228-ae32-dd2aa815eecf None None] MFA Rules not processed for user `256af8436dd842fdb4ebe90ec9be26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:46:52.279798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-904874ee-1a7a-4228-ae32-dd2aa815eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:52.302936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:46:52.366658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 649/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Wed Aug 10 16:46:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:46:52.376041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604fcfd-4b87-41c2-b4b0-7e77ea20f54d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:46:52.378447 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7604fcfd-4b87-41c2-b4b0-7e77ea20f54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:52.529946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604fcfd-4b87-41c2-b4b0-7e77ea20f54d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:46:52.618117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a64d34-61ca-476b-b87a-0f953c4dd493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:46:52.618744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64d34-61ca-476b-b87a-0f953c4dd493 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:52.619025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64d34-61ca-476b-b87a-0f953c4dd493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:52.619295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64d34-61ca-476b-b87a-0f953c4dd493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:52.620978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c4a64d34-61ca-476b-b87a-0f953c4dd493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:52.682850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:46:52.750100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a64d34-61ca-476b-b87a-0f953c4dd493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:46:52.751564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a64d34-61ca-476b-b87a-0f953c4dd493 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:46:52.773380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 662/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:46:55.147360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755e6b-0934-4f24-8b6a-ae67086be890 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:55.147878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755e6b-0934-4f24-8b6a-ae67086be890 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:55.147878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755e6b-0934-4f24-8b6a-ae67086be890 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:55.149213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 650/1312] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:46:55.154276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d938c67-0a58-4c06-8b22-10b3b905b551 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:55.154723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d938c67-0a58-4c06-8b22-10b3b905b551 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:55.154819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d938c67-0a58-4c06-8b22-10b3b905b551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:55.238930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5d938c67-0a58-4c06-8b22-10b3b905b551 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:46:55.240188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5d938c67-0a58-4c06-8b22-10b3b905b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:55.339226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 663/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:46:55.377836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e755e6b-0934-4f24-8b6a-ae67086be890 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:46:55.379445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3e755e6b-0934-4f24-8b6a-ae67086be890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:55.486570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e755e6b-0934-4f24-8b6a-ae67086be890 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:46:55.557899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:46:55.560926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9389f3-81cb-4518-9f1c-976f98509855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:46:55.561779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9389f3-81cb-4518-9f1c-976f98509855 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:55.562445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9389f3-81cb-4518-9f1c-976f98509855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:55.562662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9389f3-81cb-4518-9f1c-976f98509855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:55.564684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9a9389f3-81cb-4518-9f1c-976f98509855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:55.675370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9389f3-81cb-4518-9f1c-976f98509855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:46:55.676860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9389f3-81cb-4518-9f1c-976f98509855 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:46:55.697674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 651/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:46:58.055588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff555cc-6aba-4976-aeec-1eafda17e914 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:58.056125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff555cc-6aba-4976-aeec-1eafda17e914 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:58.056125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff555cc-6aba-4976-aeec-1eafda17e914 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:58.057638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 664/1315] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:46:58.063138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe04f2-a953-483a-8861-768c9e36fec5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:58.063596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe04f2-a953-483a-8861-768c9e36fec5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:58.063691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe04f2-a953-483a-8861-768c9e36fec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:58.138244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d6fe04f2-a953-483a-8861-768c9e36fec5 None None] MFA Rules not processed for user `9b3cf885585848eb912f6b79527201eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:46:58.139476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d6fe04f2-a953-483a-8861-768c9e36fec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:58.193454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:46:58.241929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 652/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:46:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:46:58.298824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01297c47-3562-4f0b-bb36-e15f8e55229b None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:58.299115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01297c47-3562-4f0b-bb36-e15f8e55229b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:58.299420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01297c47-3562-4f0b-bb36-e15f8e55229b None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:58.300927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 665/1317] 10.222.0.25 () {54 vars in 965 bytes} [Wed Aug 10 16:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:46:58.307410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379f5cc-c18a-41e6-83ad-cf3ee418558b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:58.307626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379f5cc-c18a-41e6-83ad-cf3ee418558b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:58.307903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379f5cc-c18a-41e6-83ad-cf3ee418558b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:58.403753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3379f5cc-c18a-41e6-83ad-cf3ee418558b None None] MFA Rules not processed for user `256af8436dd842fdb4ebe90ec9be26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:46:58.405022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3379f5cc-c18a-41e6-83ad-cf3ee418558b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:58.484711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 653/1318] 10.222.0.25 () {58 vars in 1062 bytes} [Wed Aug 10 16:46:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:46:58.497823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01297c47-3562-4f0b-bb36-e15f8e55229b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:46:58.499516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-01297c47-3562-4f0b-bb36-e15f8e55229b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:58.509381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:46:58.611363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01297c47-3562-4f0b-bb36-e15f8e55229b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:46:58.694087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920cc6b-bf66-4e08-bbc3-ba8eac520f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:46:58.694698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920cc6b-bf66-4e08-bbc3-ba8eac520f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:46:58.694973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920cc6b-bf66-4e08-bbc3-ba8eac520f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:46:58.695239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920cc6b-bf66-4e08-bbc3-ba8eac520f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:46:58.696861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5920cc6b-bf66-4e08-bbc3-ba8eac520f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:46:58.762148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:46:58.811176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920cc6b-bf66-4e08-bbc3-ba8eac520f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:46:58.812817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920cc6b-bf66-4e08-bbc3-ba8eac520f87 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:46:58.833369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 666/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:46:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:01.879290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db77e8-add0-4bd2-8ba4-66fa30fe8bf0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:01.879830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db77e8-add0-4bd2-8ba4-66fa30fe8bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:01.879830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db77e8-add0-4bd2-8ba4-66fa30fe8bf0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:01.881213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 654/1320] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:47:01.886374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846587a-140d-4c01-a753-a37d46bdde80 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:01.886774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846587a-140d-4c01-a753-a37d46bdde80 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:01.886927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846587a-140d-4c01-a753-a37d46bdde80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:01.960860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4846587a-140d-4c01-a753-a37d46bdde80 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:01.962103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4846587a-140d-4c01-a753-a37d46bdde80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:02.063351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 667/1321] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:02.103449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db77e8-add0-4bd2-8ba4-66fa30fe8bf0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:02.105047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-47db77e8-add0-4bd2-8ba4-66fa30fe8bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:02.163742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:02.217955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db77e8-add0-4bd2-8ba4-66fa30fe8bf0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:02.292106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a0dc5-c3e8-444e-bfad-a4b26d09fa95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:02.292750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0dc5-c3e8-444e-bfad-a4b26d09fa95 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:02.293035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0dc5-c3e8-444e-bfad-a4b26d09fa95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:02.293308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0dc5-c3e8-444e-bfad-a4b26d09fa95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:02.295240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-341a0dc5-c3e8-444e-bfad-a4b26d09fa95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:02.394845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:02.408123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0dc5-c3e8-444e-bfad-a4b26d09fa95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:02.410145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0dc5-c3e8-444e-bfad-a4b26d09fa95 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:02.429932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 655/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:05.180387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480497cf-385e-417d-951d-451deff11b45 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:05.180934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480497cf-385e-417d-951d-451deff11b45 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:05.180934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480497cf-385e-417d-951d-451deff11b45 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:05.182329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 668/1323] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:05.187750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bfadd-ea4c-452c-a10d-dba2c11e17cb None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:05.188316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bfadd-ea4c-452c-a10d-dba2c11e17cb None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:05.188420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bfadd-ea4c-452c-a10d-dba2c11e17cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:05.254931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c54bfadd-ea4c-452c-a10d-dba2c11e17cb None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:05.256186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c54bfadd-ea4c-452c-a10d-dba2c11e17cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:05.368010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 656/1324] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:05.405214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480497cf-385e-417d-951d-451deff11b45 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:05.411226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-480497cf-385e-417d-951d-451deff11b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:05.432600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:05.530126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480497cf-385e-417d-951d-451deff11b45 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:05.607655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87473847-e308-41f2-ac7e-5a4a1b59b5a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:05.608323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87473847-e308-41f2-ac7e-5a4a1b59b5a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:05.608609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87473847-e308-41f2-ac7e-5a4a1b59b5a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:05.608884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87473847-e308-41f2-ac7e-5a4a1b59b5a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:05.610654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-87473847-e308-41f2-ac7e-5a4a1b59b5a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:05.689651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:05.730152 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87473847-e308-41f2-ac7e-5a4a1b59b5a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:05.731507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87473847-e308-41f2-ac7e-5a4a1b59b5a0 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:05.751001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 669/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:07.839395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b91c4-4a76-452e-8f22-33810ebfeba1 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:07.839993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b91c4-4a76-452e-8f22-33810ebfeba1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:07.839993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b91c4-4a76-452e-8f22-33810ebfeba1 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:07.841348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 657/1326] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:07.848038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03847d-e75e-4cdd-abe7-f66bd86e3240 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:07.848639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03847d-e75e-4cdd-abe7-f66bd86e3240 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:07.848753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03847d-e75e-4cdd-abe7-f66bd86e3240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:07.922871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5b03847d-e75e-4cdd-abe7-f66bd86e3240 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:07.924103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5b03847d-e75e-4cdd-abe7-f66bd86e3240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:08.041536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 670/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:08.076926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b91c4-4a76-452e-8f22-33810ebfeba1 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:08.078519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0e2b91c4-4a76-452e-8f22-33810ebfeba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:08.140244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:08.187032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b91c4-4a76-452e-8f22-33810ebfeba1 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:08.265348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68beb4-be86-4f26-8c3d-737c618f6220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:08.265944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68beb4-be86-4f26-8c3d-737c618f6220 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:08.266217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68beb4-be86-4f26-8c3d-737c618f6220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:08.266484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68beb4-be86-4f26-8c3d-737c618f6220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:08.268053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7f68beb4-be86-4f26-8c3d-737c618f6220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:08.410760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:08.413046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68beb4-be86-4f26-8c3d-737c618f6220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:08.417358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68beb4-be86-4f26-8c3d-737c618f6220 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:08.437629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 658/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:10.567498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2e4c1-4fea-435a-b99b-1065d2685a65 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:10.568073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2e4c1-4fea-435a-b99b-1065d2685a65 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:10.568205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2e4c1-4fea-435a-b99b-1065d2685a65 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:10.569558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 671/1329] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:47:10.575409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e24c-33ab-43e9-954a-7b65b50f9e2c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:10.576410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e24c-33ab-43e9-954a-7b65b50f9e2c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:10.576410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e24c-33ab-43e9-954a-7b65b50f9e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:10.649419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4945e24c-33ab-43e9-954a-7b65b50f9e2c None None] MFA Rules not processed for user `9b3cf885585848eb912f6b79527201eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:10.650664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4945e24c-33ab-43e9-954a-7b65b50f9e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:10.770008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 659/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:10.808740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2e4c1-4fea-435a-b99b-1065d2685a65 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:10.810408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-28c2e4c1-4fea-435a-b99b-1065d2685a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:10.844296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:10.928358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2e4c1-4fea-435a-b99b-1065d2685a65 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:11.008920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68201ad-743e-4875-b832-9d89cd2204f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:11.009462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68201ad-743e-4875-b832-9d89cd2204f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:11.009763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68201ad-743e-4875-b832-9d89cd2204f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:11.010042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68201ad-743e-4875-b832-9d89cd2204f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:11.011671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a68201ad-743e-4875-b832-9d89cd2204f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:11.099275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:11.155101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68201ad-743e-4875-b832-9d89cd2204f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:11.157046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68201ad-743e-4875-b832-9d89cd2204f6 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:11.182038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 672/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:13.484523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2631c-a808-4786-a77f-7da906d7955c None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:13.485145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2631c-a808-4786-a77f-7da906d7955c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:13.485145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2631c-a808-4786-a77f-7da906d7955c None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:13.486505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 660/1332] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:13.492240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f31be-2fa2-4004-ba06-d3548b9a6a10 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:13.492765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f31be-2fa2-4004-ba06-d3548b9a6a10 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:13.492877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f31be-2fa2-4004-ba06-d3548b9a6a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:13.580225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-863f31be-2fa2-4004-ba06-d3548b9a6a10 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:13.582116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-863f31be-2fa2-4004-ba06-d3548b9a6a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:13.711536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 673/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:13.747819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2631c-a808-4786-a77f-7da906d7955c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:13.749402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c5c2631c-a808-4786-a77f-7da906d7955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:13.822648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:13.859125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2631c-a808-4786-a77f-7da906d7955c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:13.934244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c200c1-b595-4b17-93da-62c09f743a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:13.934859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c200c1-b595-4b17-93da-62c09f743a92 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:13.935134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c200c1-b595-4b17-93da-62c09f743a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:13.935402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c200c1-b595-4b17-93da-62c09f743a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:13.937063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-08c200c1-b595-4b17-93da-62c09f743a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:14.071510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c200c1-b595-4b17-93da-62c09f743a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:14.072927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c200c1-b595-4b17-93da-62c09f743a92 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:14.083701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:14.092461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 661/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:16.225930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293e5a-1dd0-43ca-8756-a3ddac16dfd3 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:16.226712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293e5a-1dd0-43ca-8756-a3ddac16dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:16.226712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293e5a-1dd0-43ca-8756-a3ddac16dfd3 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:16.228266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 674/1335] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:16.236220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7a1e-7013-4aa8-b170-e86ff7895b1e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:16.236721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7a1e-7013-4aa8-b170-e86ff7895b1e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:16.236821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7a1e-7013-4aa8-b170-e86ff7895b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:16.307318 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-da7f7a1e-7013-4aa8-b170-e86ff7895b1e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:16.308591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-da7f7a1e-7013-4aa8-b170-e86ff7895b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:16.421861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 662/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:16.462630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293e5a-1dd0-43ca-8756-a3ddac16dfd3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:16.464262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f2293e5a-1dd0-43ca-8756-a3ddac16dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:16.511248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:16.580744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293e5a-1dd0-43ca-8756-a3ddac16dfd3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:16.656391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c2e32-01ec-48fa-b509-5a5cd0b8cdbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:16.657013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2e32-01ec-48fa-b509-5a5cd0b8cdbc service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:16.657290 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2e32-01ec-48fa-b509-5a5cd0b8cdbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:16.657561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2e32-01ec-48fa-b509-5a5cd0b8cdbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:16.659682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1f4c2e32-01ec-48fa-b509-5a5cd0b8cdbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:16.748516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:16.777219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c2e32-01ec-48fa-b509-5a5cd0b8cdbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:16.778591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c2e32-01ec-48fa-b509-5a5cd0b8cdbc service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:16.799422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 675/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:19.189704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32f40a-0229-4def-a2b1-d0e26b76ca78 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:19.190271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32f40a-0229-4def-a2b1-d0e26b76ca78 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:19.190271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32f40a-0229-4def-a2b1-d0e26b76ca78 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:19.192135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 663/1338] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:19.197523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b0e67-871e-42d6-86b2-0923c9b3fd64 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:19.197993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b0e67-871e-42d6-86b2-0923c9b3fd64 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:19.198138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b0e67-871e-42d6-86b2-0923c9b3fd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:19.264896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-148b0e67-871e-42d6-86b2-0923c9b3fd64 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:19.266135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-148b0e67-871e-42d6-86b2-0923c9b3fd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:19.363384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 676/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:19.399873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32f40a-0229-4def-a2b1-d0e26b76ca78 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:19.401439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-cf32f40a-0229-4def-a2b1-d0e26b76ca78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:19.484917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:19.514424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32f40a-0229-4def-a2b1-d0e26b76ca78 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:19.588945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf8b3f-299b-4d3b-94b6-662cebcef244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:19.589553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8b3f-299b-4d3b-94b6-662cebcef244 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:19.589823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8b3f-299b-4d3b-94b6-662cebcef244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:19.590089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8b3f-299b-4d3b-94b6-662cebcef244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:19.591665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2ccf8b3f-299b-4d3b-94b6-662cebcef244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:19.720163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf8b3f-299b-4d3b-94b6-662cebcef244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:19.722256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf8b3f-299b-4d3b-94b6-662cebcef244 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:19.740695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 664/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:23.895773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fcdc-7f15-4dc7-bc19-d7b31e7c1f7e None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:23.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fcdc-7f15-4dc7-bc19-d7b31e7c1f7e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:23.896361 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fcdc-7f15-4dc7-bc19-d7b31e7c1f7e None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:23.898060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 677/1341] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:23.903447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee0a20-6571-4637-9381-59237c510fad None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:23.903891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee0a20-6571-4637-9381-59237c510fad None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:23.904132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee0a20-6571-4637-9381-59237c510fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:23.909802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:23.972633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c9ee0a20-6571-4637-9381-59237c510fad None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:23.973489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c9ee0a20-6571-4637-9381-59237c510fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:24.073061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 665/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:24.112911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2fcdc-7f15-4dc7-bc19-d7b31e7c1f7e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:24.114468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-89b2fcdc-7f15-4dc7-bc19-d7b31e7c1f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:24.163156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:24.222075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2fcdc-7f15-4dc7-bc19-d7b31e7c1f7e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:24.300361 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3230665-4580-4996-b09b-dfdfb1b5d413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:24.300986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3230665-4580-4996-b09b-dfdfb1b5d413 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:24.301270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3230665-4580-4996-b09b-dfdfb1b5d413 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:24.301547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3230665-4580-4996-b09b-dfdfb1b5d413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:24.303160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d3230665-4580-4996-b09b-dfdfb1b5d413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:24.395117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:24.413007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3230665-4580-4996-b09b-dfdfb1b5d413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:24.414512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3230665-4580-4996-b09b-dfdfb1b5d413 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:24.436526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 678/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:27.114711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c1490-7cf4-4634-bfa2-762889ccb588 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:27.115269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c1490-7cf4-4634-bfa2-762889ccb588 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:27.115269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c1490-7cf4-4634-bfa2-762889ccb588 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:27.116869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 666/1344] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:27.122433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9a20-8c08-415c-b1e1-e27d63dd31ff None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:27.122922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9a20-8c08-415c-b1e1-e27d63dd31ff None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:27.123087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9a20-8c08-415c-b1e1-e27d63dd31ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:27.189167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-66ab9a20-8c08-415c-b1e1-e27d63dd31ff None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:27.190876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-66ab9a20-8c08-415c-b1e1-e27d63dd31ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:27.288815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 679/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:27.326377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c1490-7cf4-4634-bfa2-762889ccb588 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:27.328251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-993c1490-7cf4-4634-bfa2-762889ccb588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:27.421207 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:27.443397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c1490-7cf4-4634-bfa2-762889ccb588 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:27.519295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf88b0-4aa7-4d9c-9632-533e63579a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:27.519908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf88b0-4aa7-4d9c-9632-533e63579a27 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:27.520213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf88b0-4aa7-4d9c-9632-533e63579a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:27.520493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf88b0-4aa7-4d9c-9632-533e63579a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:27.522078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0daf88b0-4aa7-4d9c-9632-533e63579a27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:27.653158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf88b0-4aa7-4d9c-9632-533e63579a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:27.654663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf88b0-4aa7-4d9c-9632-533e63579a27 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:27.673581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 667/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:31.945200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed41289-52a0-4cc0-928d-808d07735f1a None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:31.945736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed41289-52a0-4cc0-928d-808d07735f1a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:31.945736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed41289-52a0-4cc0-928d-808d07735f1a None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:31.947142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 680/1347] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:31.953121 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cd6f4-530b-4fe6-abd7-c4c3e1c37715 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:31.953640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cd6f4-530b-4fe6-abd7-c4c3e1c37715 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:31.953753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cd6f4-530b-4fe6-abd7-c4c3e1c37715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:31.971239 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:32.038721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-939cd6f4-530b-4fe6-abd7-c4c3e1c37715 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:32.039993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-939cd6f4-530b-4fe6-abd7-c4c3e1c37715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:32.156137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 668/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:32.197721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed41289-52a0-4cc0-928d-808d07735f1a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:32.199379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4ed41289-52a0-4cc0-928d-808d07735f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:32.261504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:32.311730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed41289-52a0-4cc0-928d-808d07735f1a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:32.391203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc6b45-ff12-4572-9095-7ae83e057083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:32.391829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc6b45-ff12-4572-9095-7ae83e057083 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:32.392114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc6b45-ff12-4572-9095-7ae83e057083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:32.392441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc6b45-ff12-4572-9095-7ae83e057083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:32.394150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aacc6b45-ff12-4572-9095-7ae83e057083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:32.507668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:32.512208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc6b45-ff12-4572-9095-7ae83e057083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:32.513678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc6b45-ff12-4572-9095-7ae83e057083 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:32.533704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 681/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:37.703148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e42225-e305-45a0-8502-3699bb73fd34 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:37.703783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e42225-e305-45a0-8502-3699bb73fd34 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:37.703783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e42225-e305-45a0-8502-3699bb73fd34 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:37.705435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 669/1350] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 16:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:37.710846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed895ed6-5eb0-4b9f-a180-b771023c3169 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:37.711286 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed895ed6-5eb0-4b9f-a180-b771023c3169 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:37.711497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed895ed6-5eb0-4b9f-a180-b771023c3169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:37.806811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ed895ed6-5eb0-4b9f-a180-b771023c3169 None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:37.808109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ed895ed6-5eb0-4b9f-a180-b771023c3169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:37.990346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 682/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 16:47:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:42.153415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=59 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:42.531528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803068c-95b4-4a0c-891e-a5e3ad383dff None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:42.531869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803068c-95b4-4a0c-891e-a5e3ad383dff None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:42.532116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803068c-95b4-4a0c-891e-a5e3ad383dff None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:42.534638 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 670/1352] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 16:47:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:42.539774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063ac99-a5bd-4ff5-aaea-048dcab2e057 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:42.540614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063ac99-a5bd-4ff5-aaea-048dcab2e057 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:42.540614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063ac99-a5bd-4ff5-aaea-048dcab2e057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:42.603894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:42.631826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-2063ac99-a5bd-4ff5-aaea-048dcab2e057 None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:42.633155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2063ac99-a5bd-4ff5-aaea-048dcab2e057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:42.758168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 683/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 16:47:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:44.153658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242712c-d6e2-427a-ad6e-0d7a1042ab9d None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:44.154196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242712c-d6e2-427a-ad6e-0d7a1042ab9d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:44.154292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242712c-d6e2-427a-ad6e-0d7a1042ab9d None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:44.155728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 671/1354] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 16:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:44.161476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4a45-4b14-4513-8c86-5134c1483023 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:44.161949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4a45-4b14-4513-8c86-5134c1483023 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:44.162041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4a45-4b14-4513-8c86-5134c1483023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:44.242003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-7cbb4a45-4b14-4513-8c86-5134c1483023 None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:44.243208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7cbb4a45-4b14-4513-8c86-5134c1483023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:44.286313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:44.353309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 684/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 16:47:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:47:48.185503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201e084-a946-4feb-b600-b0e5e80114d1 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:48.185503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201e084-a946-4feb-b600-b0e5e80114d1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:48.186983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201e084-a946-4feb-b600-b0e5e80114d1 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:48.188731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 672/1356] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:47:48.193339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66002-ae10-4f07-8068-68b40f50b470 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:48.193816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66002-ae10-4f07-8068-68b40f50b470 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:48.194227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66002-ae10-4f07-8068-68b40f50b470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:48.279758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d9b66002-ae10-4f07-8068-68b40f50b470 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:48.281387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d9b66002-ae10-4f07-8068-68b40f50b470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:48.535656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 685/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:48.590144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37fa22-ba04-42ce-83da-4a67e2997aa0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:48.590902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37fa22-ba04-42ce-83da-4a67e2997aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:48.591584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37fa22-ba04-42ce-83da-4a67e2997aa0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:48.594587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 673/1358] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 10 16:47:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:47:48.604044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f883c-2e89-4862-83c9-18c9d0f421bf None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:48.608928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f883c-2e89-4862-83c9-18c9d0f421bf None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:48.608928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f883c-2e89-4862-83c9-18c9d0f421bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:48.623529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:48.695392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-136f883c-2e89-4862-83c9-18c9d0f421bf None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:48.696794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-136f883c-2e89-4862-83c9-18c9d0f421bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:48.821531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 686/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 10 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:48.829698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37fa22-ba04-42ce-83da-4a67e2997aa0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:48.831782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ac37fa22-ba04-42ce-83da-4a67e2997aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:48.997738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37fa22-ba04-42ce-83da-4a67e2997aa0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:49.087133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:49.133117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde50d9-a0d2-449e-b58e-d9e5ab12aaf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:49.133899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde50d9-a0d2-449e-b58e-d9e5ab12aaf1 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:49.134282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde50d9-a0d2-449e-b58e-d9e5ab12aaf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:49.134660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde50d9-a0d2-449e-b58e-d9e5ab12aaf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:49.136552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-acde50d9-a0d2-449e-b58e-d9e5ab12aaf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:49.239427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde50d9-a0d2-449e-b58e-d9e5ab12aaf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:49.241734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde50d9-a0d2-449e-b58e-d9e5ab12aaf1 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:49.262085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 674/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:51.637144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e76e5c-2d0b-4fb3-bb32-3e898a909482 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:51.637970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e76e5c-2d0b-4fb3-bb32-3e898a909482 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:51.637970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e76e5c-2d0b-4fb3-bb32-3e898a909482 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:51.639344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 687/1361] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:51.645355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6ccb5-ed06-4a40-a687-87a642b244b7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:51.645843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6ccb5-ed06-4a40-a687-87a642b244b7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:51.645949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6ccb5-ed06-4a40-a687-87a642b244b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:51.716581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3fd6ccb5-ed06-4a40-a687-87a642b244b7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:51.717826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3fd6ccb5-ed06-4a40-a687-87a642b244b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:51.764481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:51.838378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 675/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:51.886904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e76e5c-2d0b-4fb3-bb32-3e898a909482 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:51.888546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a5e76e5c-2d0b-4fb3-bb32-3e898a909482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:51.989534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:52.089269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e76e5c-2d0b-4fb3-bb32-3e898a909482 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:52.234072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535863b-e0d2-4c3d-aaba-54366376b069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:52.234802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535863b-e0d2-4c3d-aaba-54366376b069 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:52.235006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535863b-e0d2-4c3d-aaba-54366376b069 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:52.235278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535863b-e0d2-4c3d-aaba-54366376b069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:52.237753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1535863b-e0d2-4c3d-aaba-54366376b069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:52.268274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:52.346444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535863b-e0d2-4c3d-aaba-54366376b069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:52.348064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535863b-e0d2-4c3d-aaba-54366376b069 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:52.376172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 688/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:54.750333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788a9f-a65f-417b-93fd-b64ea579d7cf None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:54.750880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788a9f-a65f-417b-93fd-b64ea579d7cf None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:54.751148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788a9f-a65f-417b-93fd-b64ea579d7cf None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:54.753222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 676/1364] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:54.758991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383391c-5f73-4cf3-abe7-71ef77594dbe None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:54.759877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383391c-5f73-4cf3-abe7-71ef77594dbe None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:54.760274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383391c-5f73-4cf3-abe7-71ef77594dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:54.835907 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6383391c-5f73-4cf3-abe7-71ef77594dbe None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:54.837196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6383391c-5f73-4cf3-abe7-71ef77594dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:54.940678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:54.948241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 689/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:54.990573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d941e4-5e60-4f23-aab2-312c9fa55d1e None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:54.990985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d941e4-5e60-4f23-aab2-312c9fa55d1e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:54.991263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d941e4-5e60-4f23-aab2-312c9fa55d1e None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:54.992908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 677/1366] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 10 16:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:54.998630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953684-8f92-40de-815d-b0ffbe1b5c8e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:54.999076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953684-8f92-40de-815d-b0ffbe1b5c8e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:54.999566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953684-8f92-40de-815d-b0ffbe1b5c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:55.079782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d5953684-8f92-40de-815d-b0ffbe1b5c8e None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:55.081172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d5953684-8f92-40de-815d-b0ffbe1b5c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:55.210103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 690/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 10 16:47:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:47:55.218359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d941e4-5e60-4f23-aab2-312c9fa55d1e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:55.220021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d5d941e4-5e60-4f23-aab2-312c9fa55d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:55.382866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d941e4-5e60-4f23-aab2-312c9fa55d1e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:55.404868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:55.537814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea789dd-d891-44c2-a861-5f6dd71ebb23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:55.538576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea789dd-d891-44c2-a861-5f6dd71ebb23 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:55.538958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea789dd-d891-44c2-a861-5f6dd71ebb23 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:55.539327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea789dd-d891-44c2-a861-5f6dd71ebb23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:55.541115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1ea789dd-d891-44c2-a861-5f6dd71ebb23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:55.651336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea789dd-d891-44c2-a861-5f6dd71ebb23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:55.653055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea789dd-d891-44c2-a861-5f6dd71ebb23 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:55.675067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 678/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:47:56.184273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac4f2a-7ffb-496a-aaea-51082a20eef3 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:56.184826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac4f2a-7ffb-496a-aaea-51082a20eef3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:56.184826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac4f2a-7ffb-496a-aaea-51082a20eef3 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:56.186320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 691/1369] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:47:56.192290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c70a9-9304-45c2-8838-e4332ba3d555 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:56.192774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c70a9-9304-45c2-8838-e4332ba3d555 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:56.192867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c70a9-9304-45c2-8838-e4332ba3d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:56.225498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:56.265981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8e0c70a9-9304-45c2-8838-e4332ba3d555 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:56.267376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8e0c70a9-9304-45c2-8838-e4332ba3d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:56.381137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 679/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:56.740937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d654a-42d3-40ea-aebc-5dc99ce776d5 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:56.741367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d654a-42d3-40ea-aebc-5dc99ce776d5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:56.741752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d654a-42d3-40ea-aebc-5dc99ce776d5 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:56.743333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 692/1371] 10.222.0.25 () {54 vars in 964 bytes} [Wed Aug 10 16:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:56.748270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f223b8-b9e6-4147-9bac-274757315d35 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:56.748676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f223b8-b9e6-4147-9bac-274757315d35 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:56.749112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f223b8-b9e6-4147-9bac-274757315d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:56.828903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-34f223b8-b9e6-4147-9bac-274757315d35 None None] MFA Rules not processed for user `770e3b2a34d74195bcdf1d0b6f696cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:56.830994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-34f223b8-b9e6-4147-9bac-274757315d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:56.894967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:56.931435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 680/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Wed Aug 10 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:56.939135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d654a-42d3-40ea-aebc-5dc99ce776d5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:56.940857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8b1d654a-42d3-40ea-aebc-5dc99ce776d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:56.997660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:57.061609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d654a-42d3-40ea-aebc-5dc99ce776d5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:57.142551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ae356-6ead-4061-a3d4-867b5a01b681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:57.142551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae356-6ead-4061-a3d4-867b5a01b681 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:57.142551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae356-6ead-4061-a3d4-867b5a01b681 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:57.143041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae356-6ead-4061-a3d4-867b5a01b681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:57.145771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-181ae356-6ead-4061-a3d4-867b5a01b681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:57.257087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:57.276728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ae356-6ead-4061-a3d4-867b5a01b681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:57.278726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ae356-6ead-4061-a3d4-867b5a01b681 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:57.312056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 693/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:47:57.755447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93066f27-ac29-48aa-adfe-a99ddcb4fdf3 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:57.755974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93066f27-ac29-48aa-adfe-a99ddcb4fdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:57.755974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93066f27-ac29-48aa-adfe-a99ddcb4fdf3 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:57.757322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 681/1374] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:47:57.762602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cac86-a8d9-4de7-91bc-6302a04385a5 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:57.762893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cac86-a8d9-4de7-91bc-6302a04385a5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:57.763169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cac86-a8d9-4de7-91bc-6302a04385a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:57.829686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-eb4cac86-a8d9-4de7-91bc-6302a04385a5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:47:57.831072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-eb4cac86-a8d9-4de7-91bc-6302a04385a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:57.931435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 694/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:47:57.975682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93066f27-ac29-48aa-adfe-a99ddcb4fdf3 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:47:57.977236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-93066f27-ac29-48aa-adfe-a99ddcb4fdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:58.132565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93066f27-ac29-48aa-adfe-a99ddcb4fdf3 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:47:58.185229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:47:58.248854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976484b-cb24-4caa-8604-5adc3e322cfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:47:58.249533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976484b-cb24-4caa-8604-5adc3e322cfa service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:47:58.249816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976484b-cb24-4caa-8604-5adc3e322cfa service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:47:58.250093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976484b-cb24-4caa-8604-5adc3e322cfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:47:58.251746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1976484b-cb24-4caa-8604-5adc3e322cfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:47:58.348551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976484b-cb24-4caa-8604-5adc3e322cfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:47:58.350073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976484b-cb24-4caa-8604-5adc3e322cfa service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:47:58.369616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 682/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:00.412351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d9467-80ca-4811-867e-5fec8d6f3dee None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:00.412798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d9467-80ca-4811-867e-5fec8d6f3dee None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:00.412899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d9467-80ca-4811-867e-5fec8d6f3dee None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:00.414242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 695/1377] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:00.419408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d453512-048e-4827-b4e5-d49edcafe413 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:00.419853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d453512-048e-4827-b4e5-d49edcafe413 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:00.419967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d453512-048e-4827-b4e5-d49edcafe413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:00.502674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-0d453512-048e-4827-b4e5-d49edcafe413 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:00.503934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0d453512-048e-4827-b4e5-d49edcafe413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:00.511308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:00.608197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 683/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:00.648766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d9467-80ca-4811-867e-5fec8d6f3dee None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:00.650279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3e0d9467-80ca-4811-867e-5fec8d6f3dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:00.712938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:00.801883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d9467-80ca-4811-867e-5fec8d6f3dee None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:00.974861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6516b-8f3b-4749-8f7b-ece4a58e0393 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:00.976836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6516b-8f3b-4749-8f7b-ece4a58e0393 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:00.977296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6516b-8f3b-4749-8f7b-ece4a58e0393 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:00.977723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6516b-8f3b-4749-8f7b-ece4a58e0393 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:00.979535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-03c6516b-8f3b-4749-8f7b-ece4a58e0393 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:01.000109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:01.087420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6516b-8f3b-4749-8f7b-ece4a58e0393 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:01.089064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6516b-8f3b-4749-8f7b-ece4a58e0393 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:01.108186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 696/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:03.191096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9a0e-ff8f-4485-8132-ad607e6b2820 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:03.191627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9a0e-ff8f-4485-8132-ad607e6b2820 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:03.191627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9a0e-ff8f-4485-8132-ad607e6b2820 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:03.193289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 684/1380] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:03.198682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eaf9a-8100-4a22-89ea-f98e8ef4f626 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:03.199408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eaf9a-8100-4a22-89ea-f98e8ef4f626 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:03.199408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eaf9a-8100-4a22-89ea-f98e8ef4f626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:03.279583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-390eaf9a-8100-4a22-89ea-f98e8ef4f626 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:03.280847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-390eaf9a-8100-4a22-89ea-f98e8ef4f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:03.415330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:03.420558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 697/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:03.465747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd9a0e-ff8f-4485-8132-ad607e6b2820 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:03.467364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-dbfd9a0e-ff8f-4485-8132-ad607e6b2820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:03.633625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:03.648577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd9a0e-ff8f-4485-8132-ad607e6b2820 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:03.775929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b4ca4-e67b-45a1-9ea1-277e812cfc08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:03.776487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b4ca4-e67b-45a1-9ea1-277e812cfc08 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:03.776781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b4ca4-e67b-45a1-9ea1-277e812cfc08 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:03.777048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b4ca4-e67b-45a1-9ea1-277e812cfc08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:03.778647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-389b4ca4-e67b-45a1-9ea1-277e812cfc08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:03.894831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b4ca4-e67b-45a1-9ea1-277e812cfc08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:03.896544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b4ca4-e67b-45a1-9ea1-277e812cfc08 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:03.910732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:03.921532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 685/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:06.028474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588731-1e21-45d4-8446-6aed4ad687d8 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:06.029011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588731-1e21-45d4-8446-6aed4ad687d8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:06.029011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588731-1e21-45d4-8446-6aed4ad687d8 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:06.030394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 698/1383] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:06.035811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe23854-9c9a-4d3c-8dd2-8668220a9cef None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:06.036258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe23854-9c9a-4d3c-8dd2-8668220a9cef None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:06.036399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe23854-9c9a-4d3c-8dd2-8668220a9cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:06.111366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3fe23854-9c9a-4d3c-8dd2-8668220a9cef None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:06.113166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3fe23854-9c9a-4d3c-8dd2-8668220a9cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:06.246935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 686/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:06.290371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd588731-1e21-45d4-8446-6aed4ad687d8 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:06.291897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fd588731-1e21-45d4-8446-6aed4ad687d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:06.476605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd588731-1e21-45d4-8446-6aed4ad687d8 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:06.557356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:06.607275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3430c4-4fef-42a6-99d5-df06b21a3c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:06.607995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3430c4-4fef-42a6-99d5-df06b21a3c18 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:06.608309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3430c4-4fef-42a6-99d5-df06b21a3c18 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:06.608640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3430c4-4fef-42a6-99d5-df06b21a3c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:06.610211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-af3430c4-4fef-42a6-99d5-df06b21a3c18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:06.717059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3430c4-4fef-42a6-99d5-df06b21a3c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:06.718891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3430c4-4fef-42a6-99d5-df06b21a3c18 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:06.738979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 699/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:08.759089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200848b5-5e86-4abb-92ec-a74989662f36 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:08.759684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200848b5-5e86-4abb-92ec-a74989662f36 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:08.759684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200848b5-5e86-4abb-92ec-a74989662f36 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:08.761042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 687/1386] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:08.767484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893f4cc-e21b-4c97-9207-dccd53aa3ae8 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:08.767962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893f4cc-e21b-4c97-9207-dccd53aa3ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:08.768072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893f4cc-e21b-4c97-9207-dccd53aa3ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:08.845996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-7893f4cc-e21b-4c97-9207-dccd53aa3ae8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:08.847440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7893f4cc-e21b-4c97-9207-dccd53aa3ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:08.888410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:08.970576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 700/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:09.019124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200848b5-5e86-4abb-92ec-a74989662f36 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:09.021122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-200848b5-5e86-4abb-92ec-a74989662f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:09.130558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:09.204381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200848b5-5e86-4abb-92ec-a74989662f36 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:09.330018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da018c78-8799-44f1-af48-4bff1074adb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:09.330798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da018c78-8799-44f1-af48-4bff1074adb0 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:09.331199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da018c78-8799-44f1-af48-4bff1074adb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:09.331582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da018c78-8799-44f1-af48-4bff1074adb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:09.333734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-da018c78-8799-44f1-af48-4bff1074adb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:09.377735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:09.438267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da018c78-8799-44f1-af48-4bff1074adb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:09.440509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da018c78-8799-44f1-af48-4bff1074adb0 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:09.463000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 688/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:11.522220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba8d5-c3ea-45ed-a3b7-33fccd1c8e81 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:11.522783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba8d5-c3ea-45ed-a3b7-33fccd1c8e81 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:11.522783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba8d5-c3ea-45ed-a3b7-33fccd1c8e81 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:11.524115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 701/1389] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:11.529496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c189b4-58e3-4c83-add9-fc1cd0b076fe None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:11.529950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c189b4-58e3-4c83-add9-fc1cd0b076fe None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:11.530045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c189b4-58e3-4c83-add9-fc1cd0b076fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:11.598514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c3c189b4-58e3-4c83-add9-fc1cd0b076fe None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:11.599724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c3c189b4-58e3-4c83-add9-fc1cd0b076fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:11.695724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 689/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:11.741311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdba8d5-c3ea-45ed-a3b7-33fccd1c8e81 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:11.742868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9bdba8d5-c3ea-45ed-a3b7-33fccd1c8e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:11.891676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdba8d5-c3ea-45ed-a3b7-33fccd1c8e81 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:11.908065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:12.009619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1fcd2-feac-498e-88cf-5f0c0be526ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:12.010156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1fcd2-feac-498e-88cf-5f0c0be526ee service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:12.010436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1fcd2-feac-498e-88cf-5f0c0be526ee service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:12.010704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1fcd2-feac-498e-88cf-5f0c0be526ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:12.012354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4ba1fcd2-feac-498e-88cf-5f0c0be526ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:12.106234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1fcd2-feac-498e-88cf-5f0c0be526ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:12.107820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1fcd2-feac-498e-88cf-5f0c0be526ee service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:12.126313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 702/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:48:12.925760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ee50c-142f-4c02-b134-fcb7fddb7c40 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:12.926228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ee50c-142f-4c02-b134-fcb7fddb7c40 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:12.926330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ee50c-142f-4c02-b134-fcb7fddb7c40 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:12.927671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 690/1392] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:12.932945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1460b-ac50-4982-aa67-460dda6ada06 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:12.933377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1460b-ac50-4982-aa67-460dda6ada06 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:12.933484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1460b-ac50-4982-aa67-460dda6ada06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:12.961007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:13.002456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3ed1460b-ac50-4982-aa67-460dda6ada06 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:13.003669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3ed1460b-ac50-4982-aa67-460dda6ada06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:13.099880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 703/1393] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:13.158063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678c93-d326-4102-b6d5-2b2f366209dd None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:13.158830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678c93-d326-4102-b6d5-2b2f366209dd None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:13.159116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678c93-d326-4102-b6d5-2b2f366209dd None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:13.160526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 691/1394] 10.222.0.25 () {54 vars in 967 bytes} [Wed Aug 10 16:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:13.165625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a7ee-66c2-4ae2-a597-8ac01efcd652 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:13.165858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a7ee-66c2-4ae2-a597-8ac01efcd652 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:13.166136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a7ee-66c2-4ae2-a597-8ac01efcd652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:13.246784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9d65a7ee-66c2-4ae2-a597-8ac01efcd652 None None] MFA Rules not processed for user `88c76a4a9747469d88a0a1a5748452a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:13.248000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9d65a7ee-66c2-4ae2-a597-8ac01efcd652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:13.292796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:13.328119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 704/1395] 10.222.0.25 () {58 vars in 1064 bytes} [Wed Aug 10 16:48:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:48:13.336580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb678c93-d326-4102-b6d5-2b2f366209dd None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:13.338090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-eb678c93-d326-4102-b6d5-2b2f366209dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:13.352950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:13.447353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb678c93-d326-4102-b6d5-2b2f366209dd None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:13.525814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f8318-11a4-4c23-866d-72f02d5a2657 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c76a4a9747469d88a0a1a5748452a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:13.526428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f8318-11a4-4c23-866d-72f02d5a2657 service glance] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:13.526706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f8318-11a4-4c23-866d-72f02d5a2657 service glance] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:13.527064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f8318-11a4-4c23-866d-72f02d5a2657 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:13.528694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f37f8318-11a4-4c23-866d-72f02d5a2657 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:13.603693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:13.647527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f8318-11a4-4c23-866d-72f02d5a2657 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:13.649012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f8318-11a4-4c23-866d-72f02d5a2657 service glance] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:13.667100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 692/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:14.202404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ecbed-a18e-490e-aed4-ecf96b14a274 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:14.202888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ecbed-a18e-490e-aed4-ecf96b14a274 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:14.202980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ecbed-a18e-490e-aed4-ecf96b14a274 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:14.204636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 705/1397] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:14.210032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:14.210466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:14.210768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:14.283736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:14.285025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:14.389863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 693/1398] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:14.404521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa002e-170c-4a3f-ad60-3c28f0dfa72d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:14.404824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa002e-170c-4a3f-ad60-3c28f0dfa72d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:14.405093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa002e-170c-4a3f-ad60-3c28f0dfa72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:14.435550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:14.436778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:14.466402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:14.478150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3daa002e-170c-4a3f-ad60-3c28f0dfa72d None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:14.479386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3daa002e-170c-4a3f-ad60-3c28f0dfa72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:14.557992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edbfc1-b365-437b-b926-dabc9b70113d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:14.568212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 706/1399] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:48:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:14.613095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d3e4a-9318-451a-a02f-fd0088acb9f9 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:14.613386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d3e4a-9318-451a-a02f-fd0088acb9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:14.613659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d3e4a-9318-451a-a02f-fd0088acb9f9 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:14.615039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 707/1400] 10.222.0.25 () {54 vars in 943 bytes} [Wed Aug 10 16:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:48:14.621267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960f7f4-1071-41ca-a0c3-cab09be30e95 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:14.621868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960f7f4-1071-41ca-a0c3-cab09be30e95 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:14.622152 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960f7f4-1071-41ca-a0c3-cab09be30e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:14.686498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:14.687116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:14.687394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:14.687669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:14.689363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:14.699774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:14.709764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4960f7f4-1071-41ca-a0c3-cab09be30e95 None None] MFA Rules not processed for user `f64c104787604ea4b22e2af2507b3f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:14.709992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:14.711660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4960f7f4-1071-41ca-a0c3-cab09be30e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:14.791557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:14.793186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:14.811991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 694/1401] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:48:14.833483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 708/1402] 10.222.0.25 () {58 vars in 1040 bytes} [Wed Aug 10 16:48:14 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:48:14.843233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:14.844542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:14.953317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dc3c8-5260-4ead-82c5-f421c4b8ddda service nova] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:15.017413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:15.069712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500f546-f716-43ad-b61c-02098ab40426 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:15.070319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500f546-f716-43ad-b61c-02098ab40426 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:15.070594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500f546-f716-43ad-b61c-02098ab40426 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:15.070865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500f546-f716-43ad-b61c-02098ab40426 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:15.072521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3500f546-f716-43ad-b61c-02098ab40426 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:15.157274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500f546-f716-43ad-b61c-02098ab40426 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:15.158948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500f546-f716-43ad-b61c-02098ab40426 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:15.167204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 695/1403] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:48:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:48:16.931108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fd431-57fb-4fd7-949b-b86b6c83fea3 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:16.932072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fd431-57fb-4fd7-949b-b86b6c83fea3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:16.932473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fd431-57fb-4fd7-949b-b86b6c83fea3 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:16.934354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 709/1404] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:16.942096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc930-812b-411a-8b7c-c1f8230aaa32 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:16.942680 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc930-812b-411a-8b7c-c1f8230aaa32 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:16.942770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc930-812b-411a-8b7c-c1f8230aaa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:17.022045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2e3dc930-812b-411a-8b7c-c1f8230aaa32 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:17.023408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2e3dc930-812b-411a-8b7c-c1f8230aaa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:17.071962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:17.144195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 696/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:17.186515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fd431-57fb-4fd7-949b-b86b6c83fea3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:17.188360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ed6fd431-57fb-4fd7-949b-b86b6c83fea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:17.333705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:17.381623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fd431-57fb-4fd7-949b-b86b6c83fea3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:17.523572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8800c8-8ad3-4bce-a891-d140c368408f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:17.524362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8800c8-8ad3-4bce-a891-d140c368408f service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:17.524646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8800c8-8ad3-4bce-a891-d140c368408f service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:17.524921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8800c8-8ad3-4bce-a891-d140c368408f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:17.526754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aa8800c8-8ad3-4bce-a891-d140c368408f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:17.622421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:17.637638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8800c8-8ad3-4bce-a891-d140c368408f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:17.639253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8800c8-8ad3-4bce-a891-d140c368408f service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:17.664357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 710/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:19.763998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abb245-6b39-40c9-bbbd-04d079224ad0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:19.765065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abb245-6b39-40c9-bbbd-04d079224ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:19.765647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abb245-6b39-40c9-bbbd-04d079224ad0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:19.767137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 697/1407] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:19.773130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c8b0-e02d-43b8-8924-bab43d812837 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:19.773964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c8b0-e02d-43b8-8924-bab43d812837 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:19.774464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c8b0-e02d-43b8-8924-bab43d812837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:19.846894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-7199c8b0-e02d-43b8-8924-bab43d812837 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:19.848189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7199c8b0-e02d-43b8-8924-bab43d812837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:19.956062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 711/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:20.001033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abb245-6b39-40c9-bbbd-04d079224ad0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:20.003223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d8abb245-6b39-40c9-bbbd-04d079224ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:20.170644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abb245-6b39-40c9-bbbd-04d079224ad0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:20.199154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:20.319296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22473160-45af-46cc-b85e-654f042f1a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:20.319296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22473160-45af-46cc-b85e-654f042f1a58 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:20.319296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22473160-45af-46cc-b85e-654f042f1a58 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:20.319296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22473160-45af-46cc-b85e-654f042f1a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:20.319939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-22473160-45af-46cc-b85e-654f042f1a58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:20.440599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22473160-45af-46cc-b85e-654f042f1a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:20.442342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22473160-45af-46cc-b85e-654f042f1a58 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:20.463842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 698/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:29.954219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=76 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:30.461122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=87 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:37.798715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a4ed-9e5c-41f5-a396-45e7ba75cb32 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:37.799296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a4ed-9e5c-41f5-a396-45e7ba75cb32 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:37.799667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a4ed-9e5c-41f5-a396-45e7ba75cb32 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:37.802078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 712/1410] 10.222.0.25 () {54 vars in 918 bytes} [Wed Aug 10 16:48:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:37.809059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12026a6b-2fcd-46ed-9cf9-9d3adbcee265 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:37.809701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12026a6b-2fcd-46ed-9cf9-9d3adbcee265 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:37.809701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12026a6b-2fcd-46ed-9cf9-9d3adbcee265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:37.899001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-12026a6b-2fcd-46ed-9cf9-9d3adbcee265 None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:37.901054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-12026a6b-2fcd-46ed-9cf9-9d3adbcee265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:38.028030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 699/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Aug 10 16:48:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:38.042966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc80720-dcbb-4195-946e-33626384bf00 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:38.043259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc80720-dcbb-4195-946e-33626384bf00 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:38.043565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc80720-dcbb-4195-946e-33626384bf00 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:38.044949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 713/1412] 10.222.0.25 () {54 vars in 918 bytes} [Wed Aug 10 16:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:38.050368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750891a-650d-41af-aad0-b9dc922cabbd None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:38.050594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750891a-650d-41af-aad0-b9dc922cabbd None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:38.050870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750891a-650d-41af-aad0-b9dc922cabbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:38.130100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:38.133198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b750891a-650d-41af-aad0-b9dc922cabbd None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:38.135048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b750891a-650d-41af-aad0-b9dc922cabbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:38.254446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 700/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Aug 10 16:48:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:40.782955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad0427-f446-4ba7-9713-bb4344e1838a None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:40.783522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad0427-f446-4ba7-9713-bb4344e1838a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:40.783522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad0427-f446-4ba7-9713-bb4344e1838a None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:40.784877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:40.790506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30270be-a088-4a5e-8380-94d5ac10dd2e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:40.790993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30270be-a088-4a5e-8380-94d5ac10dd2e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:40.791096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30270be-a088-4a5e-8380-94d5ac10dd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:40.871588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b30270be-a088-4a5e-8380-94d5ac10dd2e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:40.872863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b30270be-a088-4a5e-8380-94d5ac10dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:40.936670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:40.981661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:41.026575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad0427-f446-4ba7-9713-bb4344e1838a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:41.028124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-92ad0427-f446-4ba7-9713-bb4344e1838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:41.199624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad0427-f446-4ba7-9713-bb4344e1838a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:41.284733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:41.328825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38393998-99f7-4edc-ae37-5470bbee39d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:41.329730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38393998-99f7-4edc-ae37-5470bbee39d1 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:41.330579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38393998-99f7-4edc-ae37-5470bbee39d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:41.330856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38393998-99f7-4edc-ae37-5470bbee39d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:41.332529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-38393998-99f7-4edc-ae37-5470bbee39d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:41.435344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38393998-99f7-4edc-ae37-5470bbee39d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:41.437095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38393998-99f7-4edc-ae37-5470bbee39d1 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:41.458788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 715/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:42.406870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980836e-4ef2-45ca-956d-48095e70ea91 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:42.407458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980836e-4ef2-45ca-956d-48095e70ea91 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:42.407458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980836e-4ef2-45ca-956d-48095e70ea91 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:42.408922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:48:42.414375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23700bf2-32e0-4909-8e5f-7fd1eb205234 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:42.414848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23700bf2-32e0-4909-8e5f-7fd1eb205234 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:42.415127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23700bf2-32e0-4909-8e5f-7fd1eb205234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:42.498190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-23700bf2-32e0-4909-8e5f-7fd1eb205234 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:42.503446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-23700bf2-32e0-4909-8e5f-7fd1eb205234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:42.530505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:42.603049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:42.756803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980836e-4ef2-45ca-956d-48095e70ea91 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:42.758376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3980836e-4ef2-45ca-956d-48095e70ea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:42.918767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980836e-4ef2-45ca-956d-48095e70ea91 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:42.962512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:43.045422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08afe15-e224-47e6-8bd0-deeceab7e57f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:43.045948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08afe15-e224-47e6-8bd0-deeceab7e57f service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:43.046222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08afe15-e224-47e6-8bd0-deeceab7e57f service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:43.046486 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08afe15-e224-47e6-8bd0-deeceab7e57f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:43.048097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e08afe15-e224-47e6-8bd0-deeceab7e57f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:43.152063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08afe15-e224-47e6-8bd0-deeceab7e57f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:43.153742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08afe15-e224-47e6-8bd0-deeceab7e57f service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:43.181212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 703/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:44.097792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2a15c-d0e3-4a6f-b434-5b5244ce0f4d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:44.098386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2a15c-d0e3-4a6f-b434-5b5244ce0f4d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:44.098386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2a15c-d0e3-4a6f-b434-5b5244ce0f4d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:44.099701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:44.105274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14caa0b-0cf4-4ade-a670-fc2565520a1c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:44.105774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14caa0b-0cf4-4ade-a670-fc2565520a1c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:44.105884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14caa0b-0cf4-4ade-a670-fc2565520a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:44.171528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:44.196949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c14caa0b-0cf4-4ade-a670-fc2565520a1c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:44.198187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c14caa0b-0cf4-4ade-a670-fc2565520a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:44.311362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:44.472876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2a15c-d0e3-4a6f-b434-5b5244ce0f4d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:44.474407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bad2a15c-d0e3-4a6f-b434-5b5244ce0f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:44.644571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2a15c-d0e3-4a6f-b434-5b5244ce0f4d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:44.657471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:44.774588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477512-7e99-4f6c-bf7d-a88cbe90d6c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:44.775211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477512-7e99-4f6c-bf7d-a88cbe90d6c2 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:44.775493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477512-7e99-4f6c-bf7d-a88cbe90d6c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:44.775796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477512-7e99-4f6c-bf7d-a88cbe90d6c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:44.777475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b5477512-7e99-4f6c-bf7d-a88cbe90d6c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:44.908747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477512-7e99-4f6c-bf7d-a88cbe90d6c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:44.910355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477512-7e99-4f6c-bf7d-a88cbe90d6c2 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:44.929294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 718/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:45.817795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831fc5a-449f-4d5b-8535-02a7fd62790e None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:45.818347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831fc5a-449f-4d5b-8535-02a7fd62790e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:45.818347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831fc5a-449f-4d5b-8535-02a7fd62790e None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:45.819745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:45.825100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7067be-3a92-487c-bbb3-8b69a6e2c1b4 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:45.825567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7067be-3a92-487c-bbb3-8b69a6e2c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:45.825681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7067be-3a92-487c-bbb3-8b69a6e2c1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:45.854797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:45.917328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-1b7067be-3a92-487c-bbb3-8b69a6e2c1b4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:45.918574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1b7067be-3a92-487c-bbb3-8b69a6e2c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:46.027907 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:48:46.178328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831fc5a-449f-4d5b-8535-02a7fd62790e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:46.179865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6831fc5a-449f-4d5b-8535-02a7fd62790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:46.298937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:46.337946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831fc5a-449f-4d5b-8535-02a7fd62790e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:46.472516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c60d7f-7fe3-4aca-9de0-4d6065b22fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:46.473046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c60d7f-7fe3-4aca-9de0-4d6065b22fcb service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:46.473322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c60d7f-7fe3-4aca-9de0-4d6065b22fcb service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:46.473594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c60d7f-7fe3-4aca-9de0-4d6065b22fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:46.475247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-56c60d7f-7fe3-4aca-9de0-4d6065b22fcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:46.558022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:46.572138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c60d7f-7fe3-4aca-9de0-4d6065b22fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:46.573773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c60d7f-7fe3-4aca-9de0-4d6065b22fcb service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:46.594448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 706/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:47.487255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20328952-740d-46c1-b84e-d034c5e70170 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:47.487794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20328952-740d-46c1-b84e-d034c5e70170 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:47.487794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20328952-740d-46c1-b84e-d034c5e70170 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:47.489224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:47.496169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d03534-5536-4bfb-96bf-a1a7c777a0c7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:47.496609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d03534-5536-4bfb-96bf-a1a7c777a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:47.496710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d03534-5536-4bfb-96bf-a1a7c777a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:47.585716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-51d03534-5536-4bfb-96bf-a1a7c777a0c7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:47.587157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-51d03534-5536-4bfb-96bf-a1a7c777a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:47.693458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:47.846759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20328952-740d-46c1-b84e-d034c5e70170 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:47.848322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-20328952-740d-46c1-b84e-d034c5e70170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:47.938130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:48.011775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20328952-740d-46c1-b84e-d034c5e70170 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:48.130015 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39f109-8eca-4823-8b77-fb95e116a8d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:48.130673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39f109-8eca-4823-8b77-fb95e116a8d1 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:48.130955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39f109-8eca-4823-8b77-fb95e116a8d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:48.131230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39f109-8eca-4823-8b77-fb95e116a8d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:48.132887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-de39f109-8eca-4823-8b77-fb95e116a8d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:48.183165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:48.235005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39f109-8eca-4823-8b77-fb95e116a8d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:48.236578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39f109-8eca-4823-8b77-fb95e116a8d1 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:48.255209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 721/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:49.087625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215746a-6abe-47bf-8355-36d82a34d1bb None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:49.088341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215746a-6abe-47bf-8355-36d82a34d1bb None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:49.088588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215746a-6abe-47bf-8355-36d82a34d1bb None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:49.090903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:49.096488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0f4fb-f176-48b8-bfe5-9bde87e8abc1 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:49.096958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0f4fb-f176-48b8-bfe5-9bde87e8abc1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:49.097070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0f4fb-f176-48b8-bfe5-9bde87e8abc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:49.181446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-41c0f4fb-f176-48b8-bfe5-9bde87e8abc1 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:49.182689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-41c0f4fb-f176-48b8-bfe5-9bde87e8abc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:49.295417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:49.448760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215746a-6abe-47bf-8355-36d82a34d1bb None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:49.450263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2215746a-6abe-47bf-8355-36d82a34d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:49.501065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:49.602992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215746a-6abe-47bf-8355-36d82a34d1bb None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:49.737602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3652b7b-d74f-44e2-858f-8982c3edc5f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:49.738186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652b7b-d74f-44e2-858f-8982c3edc5f6 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:49.738460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652b7b-d74f-44e2-858f-8982c3edc5f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:49.738792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652b7b-d74f-44e2-858f-8982c3edc5f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:49.740450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e3652b7b-d74f-44e2-858f-8982c3edc5f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:49.747832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:49.837173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3652b7b-d74f-44e2-858f-8982c3edc5f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:49.838794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3652b7b-d74f-44e2-858f-8982c3edc5f6 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:49.858489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 709/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:50.700003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61bde6-749f-44f4-a547-72b04503dc66 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:50.700628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61bde6-749f-44f4-a547-72b04503dc66 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:50.700628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61bde6-749f-44f4-a547-72b04503dc66 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:50.701952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:50.707816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa957-8143-465b-b8fd-9192b42de52e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:50.708348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa957-8143-465b-b8fd-9192b42de52e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:50.708462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa957-8143-465b-b8fd-9192b42de52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:50.789306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f1caa957-8143-465b-b8fd-9192b42de52e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:50.790518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f1caa957-8143-465b-b8fd-9192b42de52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:50.860109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:50.890204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:51.053882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61bde6-749f-44f4-a547-72b04503dc66 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:51.055314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0f61bde6-749f-44f4-a547-72b04503dc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:51.073454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:51.215907 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61bde6-749f-44f4-a547-72b04503dc66 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:51.307625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:51.336441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e404e5-6149-4683-b365-0fffa7137491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:51.337060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e404e5-6149-4683-b365-0fffa7137491 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:51.337340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e404e5-6149-4683-b365-0fffa7137491 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:51.337614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e404e5-6149-4683-b365-0fffa7137491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:51.339230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-18e404e5-6149-4683-b365-0fffa7137491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:51.432129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e404e5-6149-4683-b365-0fffa7137491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:51.433775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e404e5-6149-4683-b365-0fffa7137491 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:51.452812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 724/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:52.276876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e5e11-317c-478e-b6e7-3be6bd23f92d None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:52.277455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e5e11-317c-478e-b6e7-3be6bd23f92d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:52.277455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e5e11-317c-478e-b6e7-3be6bd23f92d None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:52.278876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:52.285104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31790033-c523-4c10-9b05-9be01da1e3d9 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:52.285600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31790033-c523-4c10-9b05-9be01da1e3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:52.285709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31790033-c523-4c10-9b05-9be01da1e3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:52.363574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-31790033-c523-4c10-9b05-9be01da1e3d9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:52.365478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-31790033-c523-4c10-9b05-9be01da1e3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:52.408942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:52.466210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:52.619549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e5e11-317c-478e-b6e7-3be6bd23f92d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:52.621018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f82e5e11-317c-478e-b6e7-3be6bd23f92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:52.771448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e5e11-317c-478e-b6e7-3be6bd23f92d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:52.828655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:52.891600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78996987-43af-4db5-aeee-ce2fc97bbde1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:52.892648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996987-43af-4db5-aeee-ce2fc97bbde1 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:52.892926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996987-43af-4db5-aeee-ce2fc97bbde1 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:52.893196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996987-43af-4db5-aeee-ce2fc97bbde1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:52.894820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-78996987-43af-4db5-aeee-ce2fc97bbde1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:52.988352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78996987-43af-4db5-aeee-ce2fc97bbde1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:52.989990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78996987-43af-4db5-aeee-ce2fc97bbde1 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:53.009912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 712/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:53.839992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81688-6c9b-4631-8e5b-16812713ddea None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:53.840555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81688-6c9b-4631-8e5b-16812713ddea None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:53.840555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81688-6c9b-4631-8e5b-16812713ddea None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:53.841857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:53.847830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4ac0d-213f-4fe5-a544-ad16af688c64 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:53.848370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4ac0d-213f-4fe5-a544-ad16af688c64 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:53.848492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4ac0d-213f-4fe5-a544-ad16af688c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:53.929940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-efd4ac0d-213f-4fe5-a544-ad16af688c64 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:53.931184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-efd4ac0d-213f-4fe5-a544-ad16af688c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:53.933463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:54.039944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:48:54.192369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c81688-6c9b-4631-8e5b-16812713ddea None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:54.194187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c8c81688-6c9b-4631-8e5b-16812713ddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:54.347080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c81688-6c9b-4631-8e5b-16812713ddea None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:54.377938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:54.488069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23830e05-7fc4-448a-a5dc-d31332af2cf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:54.488755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23830e05-7fc4-448a-a5dc-d31332af2cf9 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:54.489038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23830e05-7fc4-448a-a5dc-d31332af2cf9 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:54.489408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23830e05-7fc4-448a-a5dc-d31332af2cf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:54.491059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-23830e05-7fc4-448a-a5dc-d31332af2cf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:54.585725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23830e05-7fc4-448a-a5dc-d31332af2cf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:54.587335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23830e05-7fc4-448a-a5dc-d31332af2cf9 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:54.606484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 727/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:56.552739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33248f0b-9791-49ad-9846-3d44a0009feb None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:56.553318 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33248f0b-9791-49ad-9846-3d44a0009feb None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:56.553318 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33248f0b-9791-49ad-9846-3d44a0009feb None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:56.554666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:56.560529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6c10c-c8fe-4234-a705-2e027de4471c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:56.561004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6c10c-c8fe-4234-a705-2e027de4471c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:56.561095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6c10c-c8fe-4234-a705-2e027de4471c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:56.603748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:56.631749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-09c6c10c-c8fe-4234-a705-2e027de4471c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:56.633005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-09c6c10c-c8fe-4234-a705-2e027de4471c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:56.733120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:56.772837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33248f0b-9791-49ad-9846-3d44a0009feb None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:56.774634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-33248f0b-9791-49ad-9846-3d44a0009feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:56.889455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33248f0b-9791-49ad-9846-3d44a0009feb None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:56.925844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:56.977383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df01a38-293d-4bda-b0ed-1267aef2f172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:56.977991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df01a38-293d-4bda-b0ed-1267aef2f172 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:56.978264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df01a38-293d-4bda-b0ed-1267aef2f172 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:56.978531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df01a38-293d-4bda-b0ed-1267aef2f172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:56.980163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8df01a38-293d-4bda-b0ed-1267aef2f172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:57.098034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df01a38-293d-4bda-b0ed-1267aef2f172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:57.099519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df01a38-293d-4bda-b0ed-1267aef2f172 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:57.118637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 715/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:48:59.292731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e899c5-29c9-46b7-9e87-4a54d5fe8d15 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:59.293265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e899c5-29c9-46b7-9e87-4a54d5fe8d15 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:59.293265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e899c5-29c9-46b7-9e87-4a54d5fe8d15 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:59.294559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:48:59.300895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca8fae-f2da-4fe6-b7b3-2146021064e7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:59.301406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca8fae-f2da-4fe6-b7b3-2146021064e7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:59.301507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca8fae-f2da-4fe6-b7b3-2146021064e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:59.374045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-beca8fae-f2da-4fe6-b7b3-2146021064e7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:48:59.374045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-beca8fae-f2da-4fe6-b7b3-2146021064e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:59.387223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:59.477681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:48:59.519723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e899c5-29c9-46b7-9e87-4a54d5fe8d15 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:48:59.521307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-40e899c5-29c9-46b7-9e87-4a54d5fe8d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:59.634504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e899c5-29c9-46b7-9e87-4a54d5fe8d15 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:48:59.647382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:48:59.717354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aac1ac-ddb8-4f8c-bb5b-f05203ed1eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:48:59.717962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aac1ac-ddb8-4f8c-bb5b-f05203ed1eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:48:59.718238 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aac1ac-ddb8-4f8c-bb5b-f05203ed1eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:48:59.718535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aac1ac-ddb8-4f8c-bb5b-f05203ed1eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:48:59.720692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-93aac1ac-ddb8-4f8c-bb5b-f05203ed1eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:48:59.851968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aac1ac-ddb8-4f8c-bb5b-f05203ed1eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:48:59.853426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aac1ac-ddb8-4f8c-bb5b-f05203ed1eb8 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:48:59.872446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 730/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:02.272826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935384ce-1be1-4d0e-8fad-ee6d1ba0358b None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:02.273413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935384ce-1be1-4d0e-8fad-ee6d1ba0358b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:02.273413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935384ce-1be1-4d0e-8fad-ee6d1ba0358b None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:02.274747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:02.280260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ddc56-f1f7-4a1f-a2d0-53119f3b6716 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:02.280718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ddc56-f1f7-4a1f-a2d0-53119f3b6716 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:02.280842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ddc56-f1f7-4a1f-a2d0-53119f3b6716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:02.331665 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:02.351623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b17ddc56-f1f7-4a1f-a2d0-53119f3b6716 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:02.352901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b17ddc56-f1f7-4a1f-a2d0-53119f3b6716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:02.461568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:02.501264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935384ce-1be1-4d0e-8fad-ee6d1ba0358b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:02.502844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-935384ce-1be1-4d0e-8fad-ee6d1ba0358b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:02.621167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935384ce-1be1-4d0e-8fad-ee6d1ba0358b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:02.665544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:02.702141 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb9340-e4e4-44b4-b5d9-ddd61ed2e8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:02.702754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9340-e4e4-44b4-b5d9-ddd61ed2e8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:02.703028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9340-e4e4-44b4-b5d9-ddd61ed2e8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:02.703384 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9340-e4e4-44b4-b5d9-ddd61ed2e8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:02.705244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f8cb9340-e4e4-44b4-b5d9-ddd61ed2e8a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:02.836831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb9340-e4e4-44b4-b5d9-ddd61ed2e8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:02.838265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb9340-e4e4-44b4-b5d9-ddd61ed2e8a3 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:02.858406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 718/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:05.056982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac94338-4f65-4998-a074-3627409bc84c None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:05.057820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac94338-4f65-4998-a074-3627409bc84c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:05.057996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac94338-4f65-4998-a074-3627409bc84c None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:05.059403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:05.065370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9e4a2-f01a-4bfa-b99b-46e2ba8eeaee None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:05.065910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9e4a2-f01a-4bfa-b99b-46e2ba8eeaee None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:05.065910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9e4a2-f01a-4bfa-b99b-46e2ba8eeaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:05.140764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-54e9e4a2-f01a-4bfa-b99b-46e2ba8eeaee None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:05.142098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-54e9e4a2-f01a-4bfa-b99b-46e2ba8eeaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:05.162937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:05.242968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:49:05.282722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac94338-4f65-4998-a074-3627409bc84c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:05.284306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4ac94338-4f65-4998-a074-3627409bc84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:05.394741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac94338-4f65-4998-a074-3627409bc84c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:05.420037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:05.478103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab17a5-32a9-4bcc-8294-574680a833e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:05.478703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab17a5-32a9-4bcc-8294-574680a833e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:05.478980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab17a5-32a9-4bcc-8294-574680a833e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:05.479250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab17a5-32a9-4bcc-8294-574680a833e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:05.480895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-42ab17a5-32a9-4bcc-8294-574680a833e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:05.606392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab17a5-32a9-4bcc-8294-574680a833e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:05.607751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab17a5-32a9-4bcc-8294-574680a833e1 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:05.632098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 733/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:07.768992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd16a-f926-48a3-b425-632ef9edffb0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:07.769570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd16a-f926-48a3-b425-632ef9edffb0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:07.769570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd16a-f926-48a3-b425-632ef9edffb0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:07.770870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:07.776349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408df43b-0297-4f56-b1c6-5c4e0b06e0e8 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:07.776834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408df43b-0297-4f56-b1c6-5c4e0b06e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:07.776938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408df43b-0297-4f56-b1c6-5c4e0b06e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:07.847604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:07.851041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-408df43b-0297-4f56-b1c6-5c4e0b06e0e8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:07.852656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-408df43b-0297-4f56-b1c6-5c4e0b06e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:07.963449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:08.007274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd16a-f926-48a3-b425-632ef9edffb0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:08.009773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b85fd16a-f926-48a3-b425-632ef9edffb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:08.117546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd16a-f926-48a3-b425-632ef9edffb0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:08.172658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:08.207198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db50ec-3449-426e-9188-84c50505e86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:08.207813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db50ec-3449-426e-9188-84c50505e86e service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:08.208089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db50ec-3449-426e-9188-84c50505e86e service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:08.208402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db50ec-3449-426e-9188-84c50505e86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:08.210023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-36db50ec-3449-426e-9188-84c50505e86e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:08.328085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db50ec-3449-426e-9188-84c50505e86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:08.329914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db50ec-3449-426e-9188-84c50505e86e service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:08.354400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 721/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:10.455631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103eeb0-b4ac-4233-874a-1a52c9a79a3a None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:10.456236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103eeb0-b4ac-4233-874a-1a52c9a79a3a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:10.456236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103eeb0-b4ac-4233-874a-1a52c9a79a3a None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:10.457570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:10.463408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176911ef-c25f-46ae-9f02-18aa2f372fae None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:10.463950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176911ef-c25f-46ae-9f02-18aa2f372fae None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:10.463950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176911ef-c25f-46ae-9f02-18aa2f372fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:10.534649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-176911ef-c25f-46ae-9f02-18aa2f372fae None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:10.535892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-176911ef-c25f-46ae-9f02-18aa2f372fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:10.567311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:10.645081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:10.684597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103eeb0-b4ac-4233-874a-1a52c9a79a3a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:10.686142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3103eeb0-b4ac-4233-874a-1a52c9a79a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:10.800017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103eeb0-b4ac-4233-874a-1a52c9a79a3a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:10.830853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:10.881618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c4a4-80d1-41a1-8f97-5257ac014fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:10.882234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c4a4-80d1-41a1-8f97-5257ac014fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:10.882513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c4a4-80d1-41a1-8f97-5257ac014fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:10.882784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c4a4-80d1-41a1-8f97-5257ac014fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:10.884420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ee79c4a4-80d1-41a1-8f97-5257ac014fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:11.010200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c4a4-80d1-41a1-8f97-5257ac014fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:11.011642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c4a4-80d1-41a1-8f97-5257ac014fb6 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:11.032581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 736/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:13.148392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179141e-90a1-413b-b5d6-e88ef617218b None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:13.149053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179141e-90a1-413b-b5d6-e88ef617218b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:13.149053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179141e-90a1-413b-b5d6-e88ef617218b None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:13.150411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:13.156443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec40d-3d7a-4f03-9896-cbb95c22bc0a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:13.156930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec40d-3d7a-4f03-9896-cbb95c22bc0a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:13.157034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec40d-3d7a-4f03-9896-cbb95c22bc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:13.225233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-142ec40d-3d7a-4f03-9896-cbb95c22bc0a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:13.226525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-142ec40d-3d7a-4f03-9896-cbb95c22bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:13.239354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:13.341396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:13.379647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179141e-90a1-413b-b5d6-e88ef617218b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:13.381280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4179141e-90a1-413b-b5d6-e88ef617218b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:13.496757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179141e-90a1-413b-b5d6-e88ef617218b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:13.558876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:13.584163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d722783-a59a-40fe-90b4-b81bbf285b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:13.584785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d722783-a59a-40fe-90b4-b81bbf285b66 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:13.585193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d722783-a59a-40fe-90b4-b81bbf285b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:13.585411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d722783-a59a-40fe-90b4-b81bbf285b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:13.587040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0d722783-a59a-40fe-90b4-b81bbf285b66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:13.704854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d722783-a59a-40fe-90b4-b81bbf285b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:13.706309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d722783-a59a-40fe-90b4-b81bbf285b66 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:13.731142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 724/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:15.831918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab100-75ed-4341-a615-ee2b7fee62b9 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:15.832526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab100-75ed-4341-a615-ee2b7fee62b9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:15.832526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab100-75ed-4341-a615-ee2b7fee62b9 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:15.833862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:15.839293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aa6a-cfc0-4ea6-85e5-aba46198c55d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:15.839691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aa6a-cfc0-4ea6-85e5-aba46198c55d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:15.839839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aa6a-cfc0-4ea6-85e5-aba46198c55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:15.911926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a216aa6a-cfc0-4ea6-85e5-aba46198c55d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:15.913192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a216aa6a-cfc0-4ea6-85e5-aba46198c55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:15.953962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:16.016795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:16.055131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab100-75ed-4341-a615-ee2b7fee62b9 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:16.056737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8a2ab100-75ed-4341-a615-ee2b7fee62b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:16.166139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab100-75ed-4341-a615-ee2b7fee62b9 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:16.207043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:16.246682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56561e7-da2e-4ea0-a58a-a441fd83a935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:16.247330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56561e7-da2e-4ea0-a58a-a441fd83a935 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:16.247614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56561e7-da2e-4ea0-a58a-a441fd83a935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:16.247888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56561e7-da2e-4ea0-a58a-a441fd83a935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:16.249546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b56561e7-da2e-4ea0-a58a-a441fd83a935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:16.368658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56561e7-da2e-4ea0-a58a-a441fd83a935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:16.370146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56561e7-da2e-4ea0-a58a-a441fd83a935 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:16.390582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 739/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:18.506499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765c51-cc54-42bb-bef2-6a0ce2b554b5 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:18.507078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765c51-cc54-42bb-bef2-6a0ce2b554b5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:18.507078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765c51-cc54-42bb-bef2-6a0ce2b554b5 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:18.508434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:18.513959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e64dc-6180-4cc8-b0b8-c24518308cd7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:18.514452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e64dc-6180-4cc8-b0b8-c24518308cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:18.514591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e64dc-6180-4cc8-b0b8-c24518308cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:18.585049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a85e64dc-6180-4cc8-b0b8-c24518308cd7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:18.586261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a85e64dc-6180-4cc8-b0b8-c24518308cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:18.604979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:18.705915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:49:18.750436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7765c51-cc54-42bb-bef2-6a0ce2b554b5 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:18.752664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d7765c51-cc54-42bb-bef2-6a0ce2b554b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:18.862273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7765c51-cc54-42bb-bef2-6a0ce2b554b5 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:18.942438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:18.949538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee000f-5cec-4517-86d0-5fa3e7d262a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:18.950152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee000f-5cec-4517-86d0-5fa3e7d262a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:18.950425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee000f-5cec-4517-86d0-5fa3e7d262a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:18.950699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee000f-5cec-4517-86d0-5fa3e7d262a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:18.952377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-aaee000f-5cec-4517-86d0-5fa3e7d262a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:19.079350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee000f-5cec-4517-86d0-5fa3e7d262a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:19.081129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee000f-5cec-4517-86d0-5fa3e7d262a5 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:19.100514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 727/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:21.185065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5699b-d2d3-493f-a061-e2972246adc3 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:21.185695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5699b-d2d3-493f-a061-e2972246adc3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:21.185695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5699b-d2d3-493f-a061-e2972246adc3 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:21.187002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:21.195260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ee80-fc48-4acf-a9bd-b54a109bd6f6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:21.195762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ee80-fc48-4acf-a9bd-b54a109bd6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:21.195861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ee80-fc48-4acf-a9bd-b54a109bd6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:21.269033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7a30ee80-fc48-4acf-a9bd-b54a109bd6f6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:21.270162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7a30ee80-fc48-4acf-a9bd-b54a109bd6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:21.328371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:21.405064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:21.444627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5699b-d2d3-493f-a061-e2972246adc3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:21.446185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e8f5699b-d2d3-493f-a061-e2972246adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:21.574833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5699b-d2d3-493f-a061-e2972246adc3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:21.624465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:21.653218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee40c0c-f7d2-405b-8a3f-41b519ace103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:21.653835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee40c0c-f7d2-405b-8a3f-41b519ace103 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:21.654108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee40c0c-f7d2-405b-8a3f-41b519ace103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:21.654375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee40c0c-f7d2-405b-8a3f-41b519ace103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:21.655970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-eee40c0c-f7d2-405b-8a3f-41b519ace103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:21.787099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee40c0c-f7d2-405b-8a3f-41b519ace103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:21.789438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee40c0c-f7d2-405b-8a3f-41b519ace103 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:21.815264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 742/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:31.403081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=22 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:31.811845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=66 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:41.802765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1840f47-aeb5-466c-aa01-e360cea18dc8 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:41.803325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1840f47-aeb5-466c-aa01-e360cea18dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:41.803325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1840f47-aeb5-466c-aa01-e360cea18dc8 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:41.804746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 729/1471] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:41.810358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7303cc-38b3-43fb-9b7b-bbc989758a20 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:41.810996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7303cc-38b3-43fb-9b7b-bbc989758a20 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:41.810996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7303cc-38b3-43fb-9b7b-bbc989758a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:41.881981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4f7303cc-38b3-43fb-9b7b-bbc989758a20 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:41.883223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4f7303cc-38b3-43fb-9b7b-bbc989758a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:41.994124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 743/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:42.082231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759201e-ea81-4682-89e5-109efb5ae136 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:42.082522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759201e-ea81-4682-89e5-109efb5ae136 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:42.082797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759201e-ea81-4682-89e5-109efb5ae136 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:42.084200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 730/1473] 10.222.0.25 () {54 vars in 967 bytes} [Wed Aug 10 16:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:42.089104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce46ea-ecca-4c9e-88fd-4726dd3d13b7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:42.089393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce46ea-ecca-4c9e-88fd-4726dd3d13b7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:42.089667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce46ea-ecca-4c9e-88fd-4726dd3d13b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:42.171138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-28ce46ea-ecca-4c9e-88fd-4726dd3d13b7 None None] MFA Rules not processed for user `88c76a4a9747469d88a0a1a5748452a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:42.172403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-28ce46ea-ecca-4c9e-88fd-4726dd3d13b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:42.195134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:42.267048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 744/1474] 10.222.0.25 () {58 vars in 1064 bytes} [Wed Aug 10 16:49:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:42.274951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759201e-ea81-4682-89e5-109efb5ae136 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:42.276628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8759201e-ea81-4682-89e5-109efb5ae136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:42.389388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759201e-ea81-4682-89e5-109efb5ae136 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:42.476922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5612aad-5f21-4cdb-80e6-2c57ce5515ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c76a4a9747469d88a0a1a5748452a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:42.477844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5612aad-5f21-4cdb-80e6-2c57ce5515ec service glance] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:42.478280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5612aad-5f21-4cdb-80e6-2c57ce5515ec service glance] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:42.478688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5612aad-5f21-4cdb-80e6-2c57ce5515ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:42.481346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a5612aad-5f21-4cdb-80e6-2c57ce5515ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:42.549382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:42.612960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5612aad-5f21-4cdb-80e6-2c57ce5515ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:42.614568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5612aad-5f21-4cdb-80e6-2c57ce5515ec service glance] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:42.634333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 731/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:44.764491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bca12-2c55-4322-99b7-a45f2e5c5c11 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:44.765089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bca12-2c55-4322-99b7-a45f2e5c5c11 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:44.765089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bca12-2c55-4322-99b7-a45f2e5c5c11 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:44.766415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 745/1476] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:49:44.772661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d0a61-0772-4d28-865b-42ebc938e2d2 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:44.773258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d0a61-0772-4d28-865b-42ebc938e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:44.773258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d0a61-0772-4d28-865b-42ebc938e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:44.842337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-497d0a61-0772-4d28-865b-42ebc938e2d2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:44.843582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-497d0a61-0772-4d28-865b-42ebc938e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:44.946286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 732/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:44.971235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bca12-2c55-4322-99b7-a45f2e5c5c11 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:44.972232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-297bca12-2c55-4322-99b7-a45f2e5c5c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:45.105786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bca12-2c55-4322-99b7-a45f2e5c5c11 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:45.157003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:45.205328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12455f27-2c10-4daf-8611-af7449ce0fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:45.205942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12455f27-2c10-4daf-8611-af7449ce0fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:45.206221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12455f27-2c10-4daf-8611-af7449ce0fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:45.206487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12455f27-2c10-4daf-8611-af7449ce0fa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:45.207122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12455f27-2c10-4daf-8611-af7449ce0fa4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:45.208892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-12455f27-2c10-4daf-8611-af7449ce0fa4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:49:45.208892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:49:45.208892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:49:45.210036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12455f27-2c10-4daf-8611-af7449ce0fa4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:45.229608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 746/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 10 16:49:44 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:49:47.348314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ae36c-a62a-4117-bcee-be822073586b None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:47.349097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ae36c-a62a-4117-bcee-be822073586b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:47.349097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ae36c-a62a-4117-bcee-be822073586b None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:47.350229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 733/1479] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:47.355564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111820-4523-43bd-8f8f-63786116283d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:47.356269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111820-4523-43bd-8f8f-63786116283d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:47.356269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111820-4523-43bd-8f8f-63786116283d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:47.434016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-40111820-4523-43bd-8f8f-63786116283d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:47.435226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-40111820-4523-43bd-8f8f-63786116283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:47.545635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 747/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:47.590005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ae36c-a62a-4117-bcee-be822073586b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:47.591738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-888ae36c-a62a-4117-bcee-be822073586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:47.602523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:47.751480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ae36c-a62a-4117-bcee-be822073586b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:47.834598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:47.869889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f8d38-7984-418e-a20c-8317f67faef1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:47.870418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f8d38-7984-418e-a20c-8317f67faef1 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:47.870694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f8d38-7984-418e-a20c-8317f67faef1 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:47.870964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f8d38-7984-418e-a20c-8317f67faef1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:47.872632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d46f8d38-7984-418e-a20c-8317f67faef1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:47.987077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f8d38-7984-418e-a20c-8317f67faef1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:47.988754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f8d38-7984-418e-a20c-8317f67faef1 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:48.008779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 734/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:50.308193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40944-3de7-4dc4-8f44-9f02d4d41c3f None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:50.308769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40944-3de7-4dc4-8f44-9f02d4d41c3f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:50.308769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40944-3de7-4dc4-8f44-9f02d4d41c3f None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:50.310100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 748/1482] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:50.315765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4c74a-804c-48ab-a219-1f21414e7674 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:50.316203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4c74a-804c-48ab-a219-1f21414e7674 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:50.316350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4c74a-804c-48ab-a219-1f21414e7674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:50.388139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b1a4c74a-804c-48ab-a219-1f21414e7674 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:50.389404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b1a4c74a-804c-48ab-a219-1f21414e7674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:50.440482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:50.499019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 735/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:50.536832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b40944-3de7-4dc4-8f44-9f02d4d41c3f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:50.538408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c9b40944-3de7-4dc4-8f44-9f02d4d41c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:50.569227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:50.656687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b40944-3de7-4dc4-8f44-9f02d4d41c3f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:50.736841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a1c81-ced1-4d02-ac6b-4d68e4bebe35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:50.737465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1c81-ced1-4d02-ac6b-4d68e4bebe35 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:50.737744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1c81-ced1-4d02-ac6b-4d68e4bebe35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:50.738016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1c81-ced1-4d02-ac6b-4d68e4bebe35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:50.739644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-817a1c81-ced1-4d02-ac6b-4d68e4bebe35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:50.821425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:50.860121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a1c81-ced1-4d02-ac6b-4d68e4bebe35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:50.871250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a1c81-ced1-4d02-ac6b-4d68e4bebe35 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:50.884202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 749/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:53.009373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ffd62-58cb-41e1-b93d-471a4ad26e24 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:53.009933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ffd62-58cb-41e1-b93d-471a4ad26e24 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:53.010057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ffd62-58cb-41e1-b93d-471a4ad26e24 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:53.011651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 736/1485] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:53.018107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a80b9c-0c44-4908-a79a-22b6972405be None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:53.018628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a80b9c-0c44-4908-a79a-22b6972405be None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:53.018730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a80b9c-0c44-4908-a79a-22b6972405be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:53.087675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a4a80b9c-0c44-4908-a79a-22b6972405be None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:53.088933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a4a80b9c-0c44-4908-a79a-22b6972405be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:53.190781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 750/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:49:53.232211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ffd62-58cb-41e1-b93d-471a4ad26e24 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:53.233922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-849ffd62-58cb-41e1-b93d-471a4ad26e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:53.351410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ffd62-58cb-41e1-b93d-471a4ad26e24 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:53.432249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:53.434531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760cfe-5005-42dd-844e-8b1c90f6896c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:53.435428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760cfe-5005-42dd-844e-8b1c90f6896c service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:53.435813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760cfe-5005-42dd-844e-8b1c90f6896c service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:53.436354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760cfe-5005-42dd-844e-8b1c90f6896c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:53.438071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-36760cfe-5005-42dd-844e-8b1c90f6896c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:53.563156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36760cfe-5005-42dd-844e-8b1c90f6896c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:53.565012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36760cfe-5005-42dd-844e-8b1c90f6896c service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:53.589784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 737/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:55.751463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730a847-161c-4630-b710-6cedd35e15e1 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:55.752021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730a847-161c-4630-b710-6cedd35e15e1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:55.752021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730a847-161c-4630-b710-6cedd35e15e1 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:55.753429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 751/1488] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:55.758928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8eace-d085-422a-a8d5-62420992cd17 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:55.759373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8eace-d085-422a-a8d5-62420992cd17 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:55.759472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8eace-d085-422a-a8d5-62420992cd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:55.828782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2fc8eace-d085-422a-a8d5-62420992cd17 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:55.830814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2fc8eace-d085-422a-a8d5-62420992cd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:55.885693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:55.934310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 738/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:55.977122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730a847-161c-4630-b710-6cedd35e15e1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:55.978865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7730a847-161c-4630-b710-6cedd35e15e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:56.017294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:56.088108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730a847-161c-4630-b710-6cedd35e15e1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:56.163810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906734d-646e-40bc-9b2a-fdbf30d92462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:56.164468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906734d-646e-40bc-9b2a-fdbf30d92462 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:56.164752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906734d-646e-40bc-9b2a-fdbf30d92462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:56.165032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906734d-646e-40bc-9b2a-fdbf30d92462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:56.166638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1906734d-646e-40bc-9b2a-fdbf30d92462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:56.262896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:56.295397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906734d-646e-40bc-9b2a-fdbf30d92462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:56.296673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906734d-646e-40bc-9b2a-fdbf30d92462 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:56.316852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 752/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:49:58.642740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54c76-7107-4208-b77e-eb8cd233b0a1 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:58.643340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54c76-7107-4208-b77e-eb8cd233b0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:58.643340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54c76-7107-4208-b77e-eb8cd233b0a1 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:58.644681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 739/1491] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 10 16:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:49:58.650294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf51973-7593-467f-a8b3-31f741058ab7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:58.650758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf51973-7593-467f-a8b3-31f741058ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:58.650871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf51973-7593-467f-a8b3-31f741058ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:58.722175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-faf51973-7593-467f-a8b3-31f741058ab7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:49:58.723416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-faf51973-7593-467f-a8b3-31f741058ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:58.832886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 753/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 10 16:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:49:58.873285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a54c76-7107-4208-b77e-eb8cd233b0a1 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:49:58.876459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-22a54c76-7107-4208-b77e-eb8cd233b0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:59.013054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a54c76-7107-4208-b77e-eb8cd233b0a1 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:49:59.088726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472076-a259-4b7c-a638-ae28196d932e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:49:59.089340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472076-a259-4b7c-a638-ae28196d932e service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:49:59.089619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472076-a259-4b7c-a638-ae28196d932e service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:49:59.089896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472076-a259-4b7c-a638-ae28196d932e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:49:59.091493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-80472076-a259-4b7c-a638-ae28196d932e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:49:59.100742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:49:59.325515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80472076-a259-4b7c-a638-ae28196d932e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:49:59.327026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80472076-a259-4b7c-a638-ae28196d932e service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:49:59.346808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 740/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:50:08.830953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=82 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:50:09.344431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=50 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:10.136727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:10.137309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:10.137309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:10.217100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:51:10.218847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:10.323810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 754/1494] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:51:10.332281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683cb09-22b7-4173-b069-f8c8d311e6a7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:10.332812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683cb09-22b7-4173-b069-f8c8d311e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:10.332812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683cb09-22b7-4173-b069-f8c8d311e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:10.409570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3683cb09-22b7-4173-b069-f8c8d311e6a7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:51:10.410703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3683cb09-22b7-4173-b069-f8c8d311e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:10.508401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 741/1495] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:51:10.513091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:10.514339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:10.525689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafab578-0d78-449a-9e79-95e68d9e69f4 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:10.586374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:10.621189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:10.621808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:10.622092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:10.622367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:10.623016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:51:10.623323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:10.624951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:10.624951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:51:10.624951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:51:10.625842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:10.637251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 755/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:51:10.645612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683cb09-22b7-4173-b069-f8c8d311e6a7 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:10.654873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683cb09-22b7-4173-b069-f8c8d311e6a7 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:10.719057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:10.753361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:10.753966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:10.754245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:10.754549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:10.755174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:10.756952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING py.warnings [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:10.756952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: warnings.warn(msg) Aug 10 16:51:10.756952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: Aug 10 16:51:10.758147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:10.814502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 742/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:51:10 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:51:10.818784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:10.820383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:10.854977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69f9c8-1295-4c9d-ba1a-6ddd2071a64d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:10.951511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:10.952156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:10.952452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:10.952727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:10.953351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:10.954371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:10.954371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:51:10.954371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:51:10.955584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:11.000103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 756/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:51:10 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:51:11.004768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:11.016099 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:11.085046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9667dc-c4c2-4262-8d89-6db6e9d9af18 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:11.169366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:11.182713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:11.183329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:11.183616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:11.183890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:11.184583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:11.185611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING py.warnings [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:11.185611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: warnings.warn(msg) Aug 10 16:51:11.185611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: Aug 10 16:51:11.186525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:11.198962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 743/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:51:11.203680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:11.205269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:11.216906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeb5ec-85e7-463f-a58c-453a1cc80c57 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:11.242875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:11.317773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88c701-9112-4fb9-9c4d-bbd8c9ae775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:11.318323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c701-9112-4fb9-9c4d-bbd8c9ae775d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:11.318612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c701-9112-4fb9-9c4d-bbd8c9ae775d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:11.319042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c701-9112-4fb9-9c4d-bbd8c9ae775d admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:11.319782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88c701-9112-4fb9-9c4d-bbd8c9ae775d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:11.320924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-2d88c701-9112-4fb9-9c4d-bbd8c9ae775d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:11.320924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:51:11.320924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:51:11.322043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88c701-9112-4fb9-9c4d-bbd8c9ae775d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:11.337271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 757/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:51:11.342184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:11.344496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:11.382588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458ed08-2673-4190-a605-a8c549330afe admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:11.486030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d716b85-9462-44c4-bc9a-72cf5d30f62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:11.486686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d716b85-9462-44c4-bc9a-72cf5d30f62e admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:11.486963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d716b85-9462-44c4-bc9a-72cf5d30f62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:11.487232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d716b85-9462-44c4-bc9a-72cf5d30f62e admin admin] PATH_INFO: `/v3/projects/4ea9bf3ac1fb4c219cabb1ef38453c5b/users/30bbb8160cea460abcda6c44d917e67f/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:11.509152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d716b85-9462-44c4-bc9a-72cf5d30f62e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea9bf3ac1fb4c219cabb1ef38453c5b, user_id=30bbb8160cea460abcda6c44d917e67f, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:11.510180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING py.warnings [None req-2d716b85-9462-44c4-bc9a-72cf5d30f62e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:11.510180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: warnings.warn(msg) Aug 10 16:51:11.510180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: Aug 10 16:51:11.512089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d716b85-9462-44c4-bc9a-72cf5d30f62e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:11.536616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 744/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:51:11 2022] PUT /identity/v3/projects/4ea9bf3ac1fb4c219cabb1ef38453c5b/users/30bbb8160cea460abcda6c44d917e67f/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:51:11.548685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9d9ea-c3ad-4a0d-adcb-cd4604088d23 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:11.548972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9d9ea-c3ad-4a0d-adcb-cd4604088d23 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:11.549248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9d9ea-c3ad-4a0d-adcb-cd4604088d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:11.608655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-dac9d9ea-c3ad-4a0d-adcb-cd4604088d23 None None] MFA Rules not processed for user `30bbb8160cea460abcda6c44d917e67f`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:51:11.609798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-dac9d9ea-c3ad-4a0d-adcb-cd4604088d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:11.687666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 758/1502] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:51:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:51:11.701061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e41b8-2f82-4d59-b35c-8efa00afd9a5 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:11.701341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e41b8-2f82-4d59-b35c-8efa00afd9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:11.701674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e41b8-2f82-4d59-b35c-8efa00afd9a5 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:11.708117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 745/1503] 10.222.0.25 () {54 vars in 964 bytes} [Wed Aug 10 16:51:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:51:11.714519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a0e14-f522-4813-b30a-fb5fe71fdb7f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:11.714822 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a0e14-f522-4813-b30a-fb5fe71fdb7f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:11.715103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a0e14-f522-4813-b30a-fb5fe71fdb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:11.748474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:11.794692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-045a0e14-f522-4813-b30a-fb5fe71fdb7f None None] MFA Rules not processed for user `770e3b2a34d74195bcdf1d0b6f696cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:51:11.796499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-045a0e14-f522-4813-b30a-fb5fe71fdb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:11.870801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 759/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Wed Aug 10 16:51:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:51:11.879234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e41b8-2f82-4d59-b35c-8efa00afd9a5 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:11.880763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-dd7e41b8-2f82-4d59-b35c-8efa00afd9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:11.931039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:11.991842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e41b8-2f82-4d59-b35c-8efa00afd9a5 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:12.084512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:12.085149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:12.085411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:12.085689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:12.087310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:12.194331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:12.195780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:12.199546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:12.217333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 746/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:51:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:51:12.266543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b58e2-710b-4d02-bd1c-2153e2b23a0d None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:12.267373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b58e2-710b-4d02-bd1c-2153e2b23a0d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:12.267534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b58e2-710b-4d02-bd1c-2153e2b23a0d None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:12.269031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 760/1506] 10.222.0.25 () {52 vars in 830 bytes} [Wed Aug 10 16:51:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:51:12.273175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:12.274379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:12.382323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c0b28-6885-4224-8825-360230b92d0a service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:12.481460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:12.482020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:12.482323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:12.482600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] PATH_INFO: `/v3/users/30bbb8160cea460abcda6c44d917e67f` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:12.495662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30bbb8160cea460abcda6c44d917e67f)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:12.496735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING py.warnings [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:12.496735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: warnings.warn(msg) Aug 10 16:51:12.496735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: Aug 10 16:51:12.497956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:12.508383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:12.565632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30bbb8160cea460abcda6c44d917e67f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:12.579446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30bbb8160cea460abcda6c44d917e67f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:12.589387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30bbb8160cea460abcda6c44d917e67f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:12.589816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30bbb8160cea460abcda6c44d917e67f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:12.612609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30bbb8160cea460abcda6c44d917e67f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:12.618161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 747/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:51:12 2022] DELETE /identity/v3/users/30bbb8160cea460abcda6c44d917e67f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:51:12.622538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:12.622858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:12.623134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:12.681257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:51:12.682499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:12.758249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:12.772401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 761/1508] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:51:12.782374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:12.783618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:12.887034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbeeed-0b6e-42d2-b5ca-6cfcc1dbe8e9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:12.943228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:12.964428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cd0a9-87dc-4451-bbc0-915c354cc4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:12.965039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd0a9-87dc-4451-bbc0-915c354cc4aa service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:12.965319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd0a9-87dc-4451-bbc0-915c354cc4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:12.965592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd0a9-87dc-4451-bbc0-915c354cc4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:12.967194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4c9cd0a9-87dc-4451-bbc0-915c354cc4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:13.075739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cd0a9-87dc-4451-bbc0-915c354cc4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:13.077261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cd0a9-87dc-4451-bbc0-915c354cc4aa service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:13.096661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 748/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:51:13.455203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:51:13.457928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:51:13.476703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07eec49-b317-45ff-985a-bc56f8460fcb None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:51:13.578144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:51:13.578734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:51:13.578991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:51:13.579264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] PATH_INFO: `/v3/projects/4ea9bf3ac1fb4c219cabb1ef38453c5b` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:51:13.590223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea9bf3ac1fb4c219cabb1ef38453c5b)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:51:13.591262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:51:13.591262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:51:13.591262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:51:13.592498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:51:13.657065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea9bf3ac1fb4c219cabb1ef38453c5b', 'request_id': 'req-84d1065d-fa88-44ca-ba9c-60bdf85880b0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:13.663393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea9bf3ac1fb4c219cabb1ef38453c5b', 'request_id': 'req-84d1065d-fa88-44ca-ba9c-60bdf85880b0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:13.677119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea9bf3ac1fb4c219cabb1ef38453c5b', 'request_id': 'req-84d1065d-fa88-44ca-ba9c-60bdf85880b0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:13.702798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] The token cache is being invalidate because project 4ea9bf3ac1fb4c219cabb1ef38453c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:51:13.703088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:13.703518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-84d1065d-fa88-44ca-ba9c-60bdf85880b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:51:13.704968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 762/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:51:13 2022] DELETE /identity/v3/projects/4ea9bf3ac1fb4c219cabb1ef38453c5b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:51:23.094297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=62 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:51:23.700724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:52:42.718661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa246d-d25d-4aeb-846c-11601192010f None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:42.719268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa246d-d25d-4aeb-846c-11601192010f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:42.719268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa246d-d25d-4aeb-846c-11601192010f None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:42.720637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:52:42.725320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f6d4-fae5-4bbd-bf6d-4dd5ba27bacc None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:42.725723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f6d4-fae5-4bbd-bf6d-4dd5ba27bacc None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:42.725865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f6d4-fae5-4bbd-bf6d-4dd5ba27bacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:42.806113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-fc12f6d4-fae5-4bbd-bf6d-4dd5ba27bacc None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:52:42.807314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fc12f6d4-fae5-4bbd-bf6d-4dd5ba27bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:52:42.908056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:52:43.063667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa246d-d25d-4aeb-846c-11601192010f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:52:43.065771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d8fa246d-d25d-4aeb-846c-11601192010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:52:43.183734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa246d-d25d-4aeb-846c-11601192010f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:52:43.189352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d37e5-30d7-4327-82cd-0071317fcba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:52:43.189975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d37e5-30d7-4327-82cd-0071317fcba7 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:43.190252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d37e5-30d7-4327-82cd-0071317fcba7 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:43.190520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d37e5-30d7-4327-82cd-0071317fcba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:43.199030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d37e5-30d7-4327-82cd-0071317fcba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:52:43.200647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d37e5-30d7-4327-82cd-0071317fcba7 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:52:43.210681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 750/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:52:44.102273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2656e-ee27-48fd-90ce-21ae912c1d77 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:44.102819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2656e-ee27-48fd-90ce-21ae912c1d77 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:44.102940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2656e-ee27-48fd-90ce-21ae912c1d77 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:44.104329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:52:44.109169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72fa75-a9ab-4ea5-aa28-ea26df4f5742 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:44.110500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72fa75-a9ab-4ea5-aa28-ea26df4f5742 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:44.110500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72fa75-a9ab-4ea5-aa28-ea26df4f5742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:44.130949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6a72fa75-a9ab-4ea5-aa28-ea26df4f5742 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:52:44.132192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6a72fa75-a9ab-4ea5-aa28-ea26df4f5742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:52:44.141809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:52:44.225153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d436-f8c4-4515-8985-9a337235f421 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:44.225442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d436-f8c4-4515-8985-9a337235f421 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:44.225719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d436-f8c4-4515-8985-9a337235f421 None None] PATH_INFO: `/v3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:44.227000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 765/1516] 10.222.0.48 () {54 vars in 925 bytes} [Wed Aug 10 16:52:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:52:44.234292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f82363-5d24-4e16-992d-9197ade190ac None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:44.235075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f82363-5d24-4e16-992d-9197ade190ac None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:44.235355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f82363-5d24-4e16-992d-9197ade190ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:44.295659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-74f82363-5d24-4e16-992d-9197ade190ac None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:52:44.296908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-74f82363-5d24-4e16-992d-9197ade190ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:52:44.299733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d436-f8c4-4515-8985-9a337235f421 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:52:44.314584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d436-f8c4-4515-8985-9a337235f421 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:52:44.319916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14b3d9-14af-4344-9873-6466ca14e177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:52:44.320542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14b3d9-14af-4344-9873-6466ca14e177 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:44.320818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14b3d9-14af-4344-9873-6466ca14e177 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:44.321087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14b3d9-14af-4344-9873-6466ca14e177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:44.329270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14b3d9-14af-4344-9873-6466ca14e177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:52:44.330851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14b3d9-14af-4344-9873-6466ca14e177 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:52:44.341248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 766/1517] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:52:44.352348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:52:44.372333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 752/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Wed Aug 10 16:52:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:52:53.553202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53073f-5b4a-4fd3-bd71-89194de5cc7a None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:53.553712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53073f-5b4a-4fd3-bd71-89194de5cc7a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:53.553813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53073f-5b4a-4fd3-bd71-89194de5cc7a None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:53.555120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 767/1519] 10.222.0.25 () {54 vars in 961 bytes} [Wed Aug 10 16:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:52:53.560237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373a86-3348-4ffd-8da0-13f2259b488c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:53.560679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373a86-3348-4ffd-8da0-13f2259b488c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:53.560912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373a86-3348-4ffd-8da0-13f2259b488c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:53.582875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-07373a86-3348-4ffd-8da0-13f2259b488c None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:52:53.584215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-07373a86-3348-4ffd-8da0-13f2259b488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:52:53.595557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 753/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Aug 10 16:52:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:52:53.603321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53073f-5b4a-4fd3-bd71-89194de5cc7a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:52:53.616742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53073f-5b4a-4fd3-bd71-89194de5cc7a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:52:53.621885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d392d10-8f3b-4a78-ad8a-7d16d674d93e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:52:53.622479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d392d10-8f3b-4a78-ad8a-7d16d674d93e service placement] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:53.622766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d392d10-8f3b-4a78-ad8a-7d16d674d93e service placement] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:53.623041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d392d10-8f3b-4a78-ad8a-7d16d674d93e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:53.631662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d392d10-8f3b-4a78-ad8a-7d16d674d93e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:52:53.633277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d392d10-8f3b-4a78-ad8a-7d16d674d93e service placement] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:52:53.643506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 768/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:52:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:52:55.549565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3ec2-013b-404a-a170-aab1bb13a7e0 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:55.550128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3ec2-013b-404a-a170-aab1bb13a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:55.550128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3ec2-013b-404a-a170-aab1bb13a7e0 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:55.551448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:52:55.556181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306abec-2b87-477c-a700-684f0aac8f51 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:55.556621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306abec-2b87-477c-a700-684f0aac8f51 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:55.556724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306abec-2b87-477c-a700-684f0aac8f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:55.576180 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4306abec-2b87-477c-a700-684f0aac8f51 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:52:55.577407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4306abec-2b87-477c-a700-684f0aac8f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:52:55.586170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:52:55.745147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a3ec2-013b-404a-a170-aab1bb13a7e0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:52:55.759958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a3ec2-013b-404a-a170-aab1bb13a7e0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:52:55.765301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae0429-83dd-4a93-87d7-7b240d69239e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:52:55.766461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae0429-83dd-4a93-87d7-7b240d69239e service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:55.766736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae0429-83dd-4a93-87d7-7b240d69239e service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:55.767005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae0429-83dd-4a93-87d7-7b240d69239e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:55.774884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae0429-83dd-4a93-87d7-7b240d69239e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:52:55.776484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae0429-83dd-4a93-87d7-7b240d69239e service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:52:55.785925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 755/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:52:56.615536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56b1d-fe0b-430a-afb9-ddc7eb83ea42 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:56.616089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56b1d-fe0b-430a-afb9-ddc7eb83ea42 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:56.616089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56b1d-fe0b-430a-afb9-ddc7eb83ea42 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:56.617504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 16:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:52:56.622144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83dadf-e2b1-499b-b91b-28f27fb2e31f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:56.622579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83dadf-e2b1-499b-b91b-28f27fb2e31f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:56.622682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83dadf-e2b1-499b-b91b-28f27fb2e31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:56.642116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6e83dadf-e2b1-499b-b91b-28f27fb2e31f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:52:56.643297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6e83dadf-e2b1-499b-b91b-28f27fb2e31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:52:56.651159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 16:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:52:56.804803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d56b1d-fe0b-430a-afb9-ddc7eb83ea42 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:52:56.818965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d56b1d-fe0b-430a-afb9-ddc7eb83ea42 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:52:56.824477 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd2cdf-0a60-49cf-b4f2-d545d1f01d57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:52:56.825064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd2cdf-0a60-49cf-b4f2-d545d1f01d57 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:52:56.825342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd2cdf-0a60-49cf-b4f2-d545d1f01d57 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:52:56.825612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd2cdf-0a60-49cf-b4f2-d545d1f01d57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:52:56.834298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd2cdf-0a60-49cf-b4f2-d545d1f01d57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:52:56.835854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd2cdf-0a60-49cf-b4f2-d545d1f01d57 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:52:56.845476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 771/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:53:01.067700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243fee-9cc6-4f5e-9983-427cb6058a30 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:01.068285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243fee-9cc6-4f5e-9983-427cb6058a30 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:01.068285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243fee-9cc6-4f5e-9983-427cb6058a30 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:01.069659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:01.075012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff1a0-252e-4bc7-b15e-f5b4ba7af764 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:01.075394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff1a0-252e-4bc7-b15e-f5b4ba7af764 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:01.075554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff1a0-252e-4bc7-b15e-f5b4ba7af764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:01.095081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3d1ff1a0-252e-4bc7-b15e-f5b4ba7af764 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:01.096323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3d1ff1a0-252e-4bc7-b15e-f5b4ba7af764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:01.103978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:53:01.175576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd243fee-9cc6-4f5e-9983-427cb6058a30 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:01.177366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-cd243fee-9cc6-4f5e-9983-427cb6058a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:01.254511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd243fee-9cc6-4f5e-9983-427cb6058a30 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:01.258095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a2a9f-2e1e-450d-ab3d-24e598c024f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:01.258699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a2a9f-2e1e-450d-ab3d-24e598c024f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:01.258974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a2a9f-2e1e-450d-ab3d-24e598c024f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:01.259241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a2a9f-2e1e-450d-ab3d-24e598c024f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:01.269746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a2a9f-2e1e-450d-ab3d-24e598c024f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:01.271187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a2a9f-2e1e-450d-ab3d-24e598c024f7 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:01.282421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 758/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:53:03.288616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207fdd-6e4e-4748-88be-42f467cf6e74 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:03.289179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207fdd-6e4e-4748-88be-42f467cf6e74 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:03.289179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207fdd-6e4e-4748-88be-42f467cf6e74 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:03.290520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:03.295789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42fa4d-724a-40f8-8d4f-7d03f4d1529b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:03.296248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42fa4d-724a-40f8-8d4f-7d03f4d1529b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:03.296352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42fa4d-724a-40f8-8d4f-7d03f4d1529b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:03.316817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9b42fa4d-724a-40f8-8d4f-7d03f4d1529b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:03.318021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9b42fa4d-724a-40f8-8d4f-7d03f4d1529b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:03.325858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:53:03.363182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207fdd-6e4e-4748-88be-42f467cf6e74 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:03.364897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ec207fdd-6e4e-4748-88be-42f467cf6e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:03.378480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207fdd-6e4e-4748-88be-42f467cf6e74 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:03.382203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7ce62-273a-4567-b5f4-8869ad170c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:03.382815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ce62-273a-4567-b5f4-8869ad170c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:03.383091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ce62-273a-4567-b5f4-8869ad170c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:03.383366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ce62-273a-4567-b5f4-8869ad170c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:03.394002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7ce62-273a-4567-b5f4-8869ad170c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:03.395493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7ce62-273a-4567-b5f4-8869ad170c3b service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:03.405690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 774/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:53:07.184474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ec7ec-4869-4e3d-ab1b-597a2db8cfc6 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:07.185005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ec7ec-4869-4e3d-ab1b-597a2db8cfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:07.185005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ec7ec-4869-4e3d-ab1b-597a2db8cfc6 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:07.186381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:07.192161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9c598-0bab-47ae-aeb9-5045f3db4cde None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:07.192661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9c598-0bab-47ae-aeb9-5045f3db4cde None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:07.192834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9c598-0bab-47ae-aeb9-5045f3db4cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:07.200530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:07.214720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-bfe9c598-0bab-47ae-aeb9-5045f3db4cde None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:07.215946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bfe9c598-0bab-47ae-aeb9-5045f3db4cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:07.224881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:53:07.529409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ec7ec-4869-4e3d-ab1b-597a2db8cfc6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:07.531573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-339ec7ec-4869-4e3d-ab1b-597a2db8cfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:07.608542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ec7ec-4869-4e3d-ab1b-597a2db8cfc6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:07.612651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fa1fe-3544-4b9f-85a2-c42bdad8eb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:07.613248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa1fe-3544-4b9f-85a2-c42bdad8eb25 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:07.613524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa1fe-3544-4b9f-85a2-c42bdad8eb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:07.613795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa1fe-3544-4b9f-85a2-c42bdad8eb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:07.622545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fa1fe-3544-4b9f-85a2-c42bdad8eb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:07.623963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fa1fe-3544-4b9f-85a2-c42bdad8eb25 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:07.633214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 761/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:53:09.669223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1824297-526c-4baf-a333-cb0a3a271035 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:09.669768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1824297-526c-4baf-a333-cb0a3a271035 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:09.669990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1824297-526c-4baf-a333-cb0a3a271035 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:09.672568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 16:53:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:09.678164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610b76-6729-4720-8a3c-c355e3ceb478 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:09.678622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610b76-6729-4720-8a3c-c355e3ceb478 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:09.678734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610b76-6729-4720-8a3c-c355e3ceb478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:09.698882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-18610b76-6729-4720-8a3c-c355e3ceb478 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:09.700097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-18610b76-6729-4720-8a3c-c355e3ceb478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:09.708254 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 16:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:53:10.017624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1824297-526c-4baf-a333-cb0a3a271035 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:10.019352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e1824297-526c-4baf-a333-cb0a3a271035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:10.034364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1824297-526c-4baf-a333-cb0a3a271035 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:10.037866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97bf29-ef03-48d0-bda1-27aea9668fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:10.038469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97bf29-ef03-48d0-bda1-27aea9668fed service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:10.038747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97bf29-ef03-48d0-bda1-27aea9668fed service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:10.039017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97bf29-ef03-48d0-bda1-27aea9668fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:10.048835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97bf29-ef03-48d0-bda1-27aea9668fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:10.050269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97bf29-ef03-48d0-bda1-27aea9668fed service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:10.059880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 777/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:53:19.693933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:20.055641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=8 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:39.963435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d2281-5fce-4572-9476-29f344ecd95c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.964076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d2281-5fce-4572-9476-29f344ecd95c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.964076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d2281-5fce-4572-9476-29f344ecd95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.988399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-849d2281-5fce-4572-9476-29f344ecd95c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:39.989780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-849d2281-5fce-4572-9476-29f344ecd95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.001527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3376-d658-402a-b2c5-04001a60f153 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.002084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3376-d658-402a-b2c5-04001a60f153 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.002182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3376-d658-402a-b2c5-04001a60f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.002902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 763/1540] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.016771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31591073-78a2-4b37-8edb-7b8eea844443 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.017083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31591073-78a2-4b37-8edb-7b8eea844443 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.017332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31591073-78a2-4b37-8edb-7b8eea844443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.037349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b84a3376-d658-402a-b2c5-04001a60f153 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.038511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b84a3376-d658-402a-b2c5-04001a60f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.049166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-31591073-78a2-4b37-8edb-7b8eea844443 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.050717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-31591073-78a2-4b37-8edb-7b8eea844443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.059023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 778/1541] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:54:40.062326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 764/1542] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:54:40.066532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3376-d658-402a-b2c5-04001a60f153 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.080488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024350f4-b4ff-4007-9ac6-7a113481de51 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.080913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024350f4-b4ff-4007-9ac6-7a113481de51 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.081186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3376-d658-402a-b2c5-04001a60f153 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.081340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024350f4-b4ff-4007-9ac6-7a113481de51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.092814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.093543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.093832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.094109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.094773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.095079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.096724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.114781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 779/1543] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.121160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-024350f4-b4ff-4007-9ac6-7a113481de51 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.122690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-024350f4-b4ff-4007-9ac6-7a113481de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.122819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.133174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357fc2-d2c1-4ea1-a623-dd8d6aeb5fa6 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.133818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 765/1544] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 10 16:54:40.138574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024350f4-b4ff-4007-9ac6-7a113481de51 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.150057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6cdd8-998d-435a-a853-a3e7c85f54e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.156176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cdd8-998d-435a-a853-a3e7c85f54e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.156176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cdd8-998d-435a-a853-a3e7c85f54e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.156176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cdd8-998d-435a-a853-a3e7c85f54e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.156176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6cdd8-998d-435a-a853-a3e7c85f54e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.156176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-1bc6cdd8-998d-435a-a853-a3e7c85f54e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:54:40.156176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:54:40.156176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:54:40.156739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6cdd8-998d-435a-a853-a3e7c85f54e7 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.158808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024350f4-b4ff-4007-9ac6-7a113481de51 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.173698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.174776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.175280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.175748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.176786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.177130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.178217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING py.warnings [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:54:40.178217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: warnings.warn(msg) Aug 10 16:54:40.178217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: Aug 10 16:54:40.179162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d8714-ef2b-4712-9e42-3eed0020d5f0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.207737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 766/1545] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:40.210371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd14f-0395-4564-92af-3ced4e9d4433 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.210661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd14f-0395-4564-92af-3ced4e9d4433 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.210933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd14f-0395-4564-92af-3ced4e9d4433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.224044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 780/1546] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:40.225409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b427-0180-4e85-b870-5d37f83d9c5d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.225674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b427-0180-4e85-b870-5d37f83d9c5d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.225955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b427-0180-4e85-b870-5d37f83d9c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.249153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a9ffd14f-0395-4564-92af-3ced4e9d4433 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.250366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a9ffd14f-0395-4564-92af-3ced4e9d4433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.259444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a7c0b427-0180-4e85-b870-5d37f83d9c5d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.260699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a7c0b427-0180-4e85-b870-5d37f83d9c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.319795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 767/1547] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 10 16:54:40.320636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffd14f-0395-4564-92af-3ced4e9d4433 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.330167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffd14f-0395-4564-92af-3ced4e9d4433 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.334918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a4d5f-db7a-410a-9c45-8460cc378f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.335604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4d5f-db7a-410a-9c45-8460cc378f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.335893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4d5f-db7a-410a-9c45-8460cc378f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.336189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4d5f-db7a-410a-9c45-8460cc378f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.336810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a4d5f-db7a-410a-9c45-8460cc378f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.337245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 781/1548] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.338709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a4d5f-db7a-410a-9c45-8460cc378f82 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.343453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b427-0180-4e85-b870-5d37f83d9c5d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.353119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b427-0180-4e85-b870-5d37f83d9c5d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.358734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a125d-abd1-4b2e-be59-5528eccf09b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.359595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a125d-abd1-4b2e-be59-5528eccf09b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.360021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a125d-abd1-4b2e-be59-5528eccf09b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.360299 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a125d-abd1-4b2e-be59-5528eccf09b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.360930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a125d-abd1-4b2e-be59-5528eccf09b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.363225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a125d-abd1-4b2e-be59-5528eccf09b2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.368495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 768/1549] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:40.369929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6824-dae1-4dcd-9390-45cb895c5b85 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.370307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6824-dae1-4dcd-9390-45cb895c5b85 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.370783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6824-dae1-4dcd-9390-45cb895c5b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.384397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 782/1550] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:40.385628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709c70-4cb2-4233-bdad-6ec6c844b0fe None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.386054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709c70-4cb2-4233-bdad-6ec6c844b0fe None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.386336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709c70-4cb2-4233-bdad-6ec6c844b0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.392646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a7dd6824-dae1-4dcd-9390-45cb895c5b85 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.393852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a7dd6824-dae1-4dcd-9390-45cb895c5b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.406906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b0709c70-4cb2-4233-bdad-6ec6c844b0fe None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.408202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b0709c70-4cb2-4233-bdad-6ec6c844b0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.468894 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 769/1551] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.470437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec3864-0f09-4ef0-bcfc-42cff1314037 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.470727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec3864-0f09-4ef0-bcfc-42cff1314037 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.471002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec3864-0f09-4ef0-bcfc-42cff1314037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.479028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 783/1552] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.480322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1254b-b0fa-458a-9de1-e94852e83c8a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.480614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1254b-b0fa-458a-9de1-e94852e83c8a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.480890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1254b-b0fa-458a-9de1-e94852e83c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.498514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a6ec3864-0f09-4ef0-bcfc-42cff1314037 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.499721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a6ec3864-0f09-4ef0-bcfc-42cff1314037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.500104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-2bb1254b-b0fa-458a-9de1-e94852e83c8a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.501293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2bb1254b-b0fa-458a-9de1-e94852e83c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.507968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 770/1553] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.509248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 784/1554] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.509377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec4c7-449d-43a3-bdd3-ecb5db721934 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.509466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec4c7-449d-43a3-bdd3-ecb5db721934 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.509694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec4c7-449d-43a3-bdd3-ecb5db721934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.510256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f057c-1993-435a-a253-d79681cf7de7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.510537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f057c-1993-435a-a253-d79681cf7de7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.510806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f057c-1993-435a-a253-d79681cf7de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.529905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f15ec4c7-449d-43a3-bdd3-ecb5db721934 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.530997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ad1f057c-1993-435a-a253-d79681cf7de7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.531233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f15ec4c7-449d-43a3-bdd3-ecb5db721934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.532186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ad1f057c-1993-435a-a253-d79681cf7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.540354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 771/1555] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.541208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ec4c7-449d-43a3-bdd3-ecb5db721934 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.541644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 785/1556] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.542232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f057c-1993-435a-a253-d79681cf7de7 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.554529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f057c-1993-435a-a253-d79681cf7de7 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.554688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ec4c7-449d-43a3-bdd3-ecb5db721934 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.559198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b5e4e-0604-4725-aea3-8037319cf76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.559876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79daef6-1b17-4d82-ae2b-14d7247a875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.560424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b5e4e-0604-4725-aea3-8037319cf76c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.560535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79daef6-1b17-4d82-ae2b-14d7247a875e admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.560746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b5e4e-0604-4725-aea3-8037319cf76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.560857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79daef6-1b17-4d82-ae2b-14d7247a875e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.561013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b5e4e-0604-4725-aea3-8037319cf76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.561217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79daef6-1b17-4d82-ae2b-14d7247a875e admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.561832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b5e4e-0604-4725-aea3-8037319cf76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.561953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79daef6-1b17-4d82-ae2b-14d7247a875e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.562884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING py.warnings [None req-b79daef6-1b17-4d82-ae2b-14d7247a875e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:54:40.562884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: warnings.warn(msg) Aug 10 16:54:40.562884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: Aug 10 16:54:40.563432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b5e4e-0604-4725-aea3-8037319cf76c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.564078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79daef6-1b17-4d82-ae2b-14d7247a875e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.574533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 786/1557] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:40.575851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034bc2-753e-4b60-848d-7f7b88e4db81 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.576257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034bc2-753e-4b60-848d-7f7b88e4db81 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.576610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034bc2-753e-4b60-848d-7f7b88e4db81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.589418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 772/1558] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:40.591260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4301d9-92f5-443d-9300-a6f3711cdb7a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.591610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4301d9-92f5-443d-9300-a6f3711cdb7a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.591959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4301d9-92f5-443d-9300-a6f3711cdb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.598010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-95034bc2-753e-4b60-848d-7f7b88e4db81 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.599298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-95034bc2-753e-4b60-848d-7f7b88e4db81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.611045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 787/1559] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.611045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39265-0534-4e7d-bd5a-8c75140b6dbc None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.611045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39265-0534-4e7d-bd5a-8c75140b6dbc None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.611045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39265-0534-4e7d-bd5a-8c75140b6dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.612656 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-bb4301d9-92f5-443d-9300-a6f3711cdb7a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.613864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-bb4301d9-92f5-443d-9300-a6f3711cdb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.616787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.623718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 773/1560] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.625024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa80445-285d-4f02-a5d8-bb6d6d448571 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.625366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa80445-285d-4f02-a5d8-bb6d6d448571 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.625695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa80445-285d-4f02-a5d8-bb6d6d448571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.630941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-05b39265-0534-4e7d-bd5a-8c75140b6dbc None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.632133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-05b39265-0534-4e7d-bd5a-8c75140b6dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.640125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 788/1561] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.640125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39265-0534-4e7d-bd5a-8c75140b6dbc None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.648030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-daa80445-285d-4f02-a5d8-bb6d6d448571 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.649274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-daa80445-285d-4f02-a5d8-bb6d6d448571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.649972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39265-0534-4e7d-bd5a-8c75140b6dbc None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.655079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.655660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.655941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.656240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.656880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.657181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.657783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 774/1562] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:40.658454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa80445-285d-4f02-a5d8-bb6d6d448571 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.658773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.666112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.671603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 789/1563] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.672192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa80445-285d-4f02-a5d8-bb6d6d448571 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.672452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.680687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.681287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.681561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.681828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.682466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.682910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.684895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.688401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec71da-8e3b-4466-bb3b-fb4bd89f9e87 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.693239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.693834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.694114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.694386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.695048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.696731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.706387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 775/1564] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.706387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.709530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c41e9-3175-46e5-9e08-4f6949fb1512 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.714051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.714651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.714930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.715204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.715844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.716755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 790/1565] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:40.717594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.720974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.727206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048d3c1-8bcf-4b11-ad82-e2fa15cbd176 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.731532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 776/1566] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:40.731941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.732412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.732568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.733065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.733344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.733984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.734289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.735876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.742353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8065b27-0ce7-41a3-b0b3-c12924147da5 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.747475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.748060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.748445 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 791/1567] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.748885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.748994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.749255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.749947 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.750364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.751948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.759428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875abb3-1dac-46e1-b8d3-b610415c6506 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.764606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 777/1568] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.764994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.765395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.765587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.765867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.766162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.766767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.767095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.768718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.776302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731468d-db01-4281-b47d-8d7e02f99921 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.781192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.781792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.781895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 792/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.782206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.782554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.782655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.783208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.783515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.785232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.792902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4192c8-0ef3-4b86-8686-a934844401d0 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.797564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.804798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 778/1570] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.804952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.805075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.805174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.805174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.805174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.805174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.814003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98e33c-be67-4450-910e-28a519acd555 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.818775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.819536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.819816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.820125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.820875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.822812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.833257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 793/1571] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:40.833257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.856762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 779/1572] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:40.856762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.894139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523525d2-541b-4042-a697-6ae573b88af3 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.916755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2b6c2-9840-4e71-8e73-9f1e9922ffe4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.922695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.923298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.923582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.923852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.924524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.926126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.936258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 780/1573] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:40.937040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.946379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f2be-7bd8-42c8-a1e3-80998b0d7a5e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.950983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.951551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.951831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.952104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.952741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.953092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.953793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.954076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.954351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] PATH_INFO: `/v3/projects/a5cd009fd06248eebfe0c84b055ef3e0/users/7a701149974b45db9f782bc5a232aabe/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.954614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.969265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cd009fd06248eebfe0c84b055ef3e0, user_id=7a701149974b45db9f782bc5a232aabe, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.970495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:54:40.970495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:54:40.970495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:54:40.972912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.987387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 781/1574] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:40.987387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.988035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 794/1575] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:40 2022] PUT /identity/v3/projects/a5cd009fd06248eebfe0c84b055ef3e0/users/7a701149974b45db9f782bc5a232aabe/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:40.988035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.001974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.048042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26651d0-347d-44f4-896a-34d0c1b5e64a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.052201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23155b6d-f53c-49e5-8fbe-aae097966f42 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.052852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.053456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.053739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.054010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.054626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.056623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.057441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.058053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.058331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.058605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.059284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.061679 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.068491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.086609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 782/1576] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.087400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.094669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 795/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.094669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.154223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652356f-dc4e-4c8e-a49f-a9fdc973288f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.155189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2b6aa-ee19-497d-b6a0-721cf9098984 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.158972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.159588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.159868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.160068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.160224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.160735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.160858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.161010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.161276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.161885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.162723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.163751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.189718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 796/1578] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.189718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.191197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 783/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.191871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.197501 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48469aa-3074-4e9e-ac11-cc1506a2f86e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.253648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.253944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a53c67-011b-4e6e-ad96-9e19c362ca2d admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.254257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.254636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.254920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.255532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.257598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.258667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b886f-cbbb-4460-8490-014f8174c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.259323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b886f-cbbb-4460-8490-014f8174c99e admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.259608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b886f-cbbb-4460-8490-014f8174c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.259883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b886f-cbbb-4460-8490-014f8174c99e admin admin] PATH_INFO: `/v3/projects/25eeef036a7e44b0b5ee4d989f7ec507/users/6677b89e26a94e5da074431e11e12050/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.276682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b886f-cbbb-4460-8490-014f8174c99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25eeef036a7e44b0b5ee4d989f7ec507, user_id=6677b89e26a94e5da074431e11e12050, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.279003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 797/1580] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.279789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.281211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b886f-cbbb-4460-8490-014f8174c99e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.289868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670b5db-cb09-49d7-b8cd-8a36bfbd72a1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.296883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 784/1581] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:41 2022] PUT /identity/v3/projects/25eeef036a7e44b0b5ee4d989f7ec507/users/6677b89e26a94e5da074431e11e12050/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:41.298104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e21162-b13d-421e-95a8-7182864e9905 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.298391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e21162-b13d-421e-95a8-7182864e9905 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.298664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e21162-b13d-421e-95a8-7182864e9905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.308211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.320041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-15e21162-b13d-421e-95a8-7182864e9905 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:41.321330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-15e21162-b13d-421e-95a8-7182864e9905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:41.354096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.354642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.354924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.356012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.356012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.358368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.373409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.380374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 798/1582] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.381031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.387262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 785/1583] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:41.388087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e21162-b13d-421e-95a8-7182864e9905 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.393709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439ac0a-c2de-4045-9036-3d6629ee7e3b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.397706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e21162-b13d-421e-95a8-7182864e9905 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.399167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.399770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.400055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.400372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.401002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.402471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.403450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.403738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.404090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.404458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.405148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.407140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.428303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 799/1584] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.428303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.431382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 786/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.432231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.441456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecf3a8-a001-4653-9ce7-fe162264d6aa admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.443687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574b511-53d5-4781-8b81-f5bed135d384 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.446144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.446743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.447023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.447295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.447935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.448308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72873fd5-0cb0-4020-9ecd-dbce6a3f65e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.448893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72873fd5-0cb0-4020-9ecd-dbce6a3f65e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.449170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72873fd5-0cb0-4020-9ecd-dbce6a3f65e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.449437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72873fd5-0cb0-4020-9ecd-dbce6a3f65e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.449592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.450069 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72873fd5-0cb0-4020-9ecd-dbce6a3f65e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.451949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72873fd5-0cb0-4020-9ecd-dbce6a3f65e5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.463484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 800/1586] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.463484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.474528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 787/1587] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:41.477697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b73e7-9504-466a-9079-bee99a5c6ef6 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.477787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b412-8fad-4a00-81a8-557ede6dcdd9 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.477787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b412-8fad-4a00-81a8-557ede6dcdd9 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.477787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b412-8fad-4a00-81a8-557ede6dcdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.481707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.482308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.482582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.482851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.483480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.485088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.495409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-29e8b412-8fad-4a00-81a8-557ede6dcdd9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:41.498826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 801/1588] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.498826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.499059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-29e8b412-8fad-4a00-81a8-557ede6dcdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:41.504587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 788/1589] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:41.505359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8b412-8fad-4a00-81a8-557ede6dcdd9 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.509151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c538a6-8278-4668-bb61-92323e1bbef5 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.514647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.515258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.515539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.515811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.516481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.518109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.531455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 802/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.531455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.539342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da7662-2923-48ec-b83e-4cab21ac253c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.545298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.545890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.546171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.546446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.547076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.548691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.559029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 803/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.560374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.564034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8b412-8fad-4a00-81a8-557ede6dcdd9 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.569417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.570026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.570303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.570572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.570860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c338db-c8d7-4d67-b23f-ac3eb300b211 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.576495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.577090 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.577372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.577644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.578303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.579881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.580995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.582843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.590321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 804/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.591252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.596424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 789/1593] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:41.597163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.600954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8124-e20b-40a7-8d74-53fda3fd738c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.605854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.606434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.606716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.606990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.607627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.607732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc2fb-711b-4a83-a26b-54c5a8668d9d service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.609454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.613288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.614107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.614313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.614394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.615657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.617733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.621030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 805/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.622350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.629281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 790/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.630125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.633366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e4aa3-a893-4a65-9922-5b07a89c80a2 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.638958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.639571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.639866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.640139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.640811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.642417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.642578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062708e1-84d9-4139-aceb-e52217e50211 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.646871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.648115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.648575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.649015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.653296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 806/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.654172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.659884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.661514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.667056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b404a3-efa0-44ec-8585-2742ebece620 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.672746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.673437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.673785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.674066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.674573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 791/1597] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:41.674700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.675468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.676452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.685745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a095ad-7088-4a2e-9950-77ed88d8d8bd service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.688305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 807/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.689435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.694011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.694743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.695338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.695620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.696299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.698303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.702249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc4821-d698-4b6b-93f4-e5c16a051312 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.708087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.708880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.709319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.709745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.710487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 792/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.710866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.711511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.712526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.724931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046d65d-68e9-472d-bb46-0f4dee227033 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.729765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 808/1600] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.734350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.734524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.734524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.734524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.734524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] PATH_INFO: `/v3/projects/d17c2f74324146ca932ff151c9f7a003/users/96495ad2ad764ba3a0fb1e3bc28fd8bd/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.742922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9730b4-51e1-412d-8606-d84b72a5d74d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.749077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.749690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.749970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.750283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.750937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.753006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.756075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17c2f74324146ca932ff151c9f7a003, user_id=96495ad2ad764ba3a0fb1e3bc28fd8bd, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.759466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.764812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 809/1601] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:41.769422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.776025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 793/1602] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:41 2022] PUT /identity/v3/projects/d17c2f74324146ca932ff151c9f7a003/users/96495ad2ad764ba3a0fb1e3bc28fd8bd/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:41.777117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.784292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f1811-7665-4b03-9e52-020f28990687 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.830820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.853653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.854334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.854621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.854896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] PATH_INFO: `/v3/projects/f7d47026aa0a4257a64b7b1ece2a574d/users/1ff51837b33e404d91fc3754c453e99c/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.869772 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d59f2-91ce-4b78-ae47-bc4c4ea5068c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.874146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d47026aa0a4257a64b7b1ece2a574d, user_id=1ff51837b33e404d91fc3754c453e99c, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.875017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.875692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.875993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.876308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] PATH_INFO: `/v3/projects/36cf7bbfa5d3420996a2af036cf8e0da/users/c7fac765624c4e0b85b01f356a14a1cb/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.877613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.878897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.896639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 810/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:41 2022] PUT /identity/v3/projects/f7d47026aa0a4257a64b7b1ece2a574d/users/1ff51837b33e404d91fc3754c453e99c/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:41.897845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cf7bbfa5d3420996a2af036cf8e0da, user_id=c7fac765624c4e0b85b01f356a14a1cb, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.903328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.904854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.927152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 794/1604] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:41 2022] PUT /identity/v3/projects/36cf7bbfa5d3420996a2af036cf8e0da/users/c7fac765624c4e0b85b01f356a14a1cb/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:41.928357 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.990645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6e628-e0fe-4849-93b2-b3710feb6b6d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.034322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa17c0f-9efe-4285-9f62-b7f55703f1d3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.039935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.041763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.041763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.041954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] PATH_INFO: `/v3/projects/c11dfd80b6c544f6b87a58d5a25c5649/users/77b8847d10744ec98e0ed7a3f1e991ed/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.062308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c11dfd80b6c544f6b87a58d5a25c5649, user_id=77b8847d10744ec98e0ed7a3f1e991ed, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.066764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.067129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9ca71-682d-423b-ae2e-7b2b799ec525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.068014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9ca71-682d-423b-ae2e-7b2b799ec525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.068330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9ca71-682d-423b-ae2e-7b2b799ec525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.068608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9ca71-682d-423b-ae2e-7b2b799ec525 admin admin] PATH_INFO: `/v3/projects/333933f84263401dbcfee37d9078aba0/users/0f4970620f8b4a9aa7eee97a89ef56b7/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.085986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 795/1605] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:41 2022] PUT /identity/v3/projects/c11dfd80b6c544f6b87a58d5a25c5649/users/77b8847d10744ec98e0ed7a3f1e991ed/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:42.086981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.087592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9ca71-682d-423b-ae2e-7b2b799ec525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333933f84263401dbcfee37d9078aba0, user_id=0f4970620f8b4a9aa7eee97a89ef56b7, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.091587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9ca71-682d-423b-ae2e-7b2b799ec525 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.105892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 811/1606] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:41 2022] PUT /identity/v3/projects/333933f84263401dbcfee37d9078aba0/users/0f4970620f8b4a9aa7eee97a89ef56b7/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:42.107267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d697982-5eca-429a-a6f0-c0aface02695 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.107558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d697982-5eca-429a-a6f0-c0aface02695 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.107830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d697982-5eca-429a-a6f0-c0aface02695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.149594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0d697982-5eca-429a-a6f0-c0aface02695 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:42.153655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0d697982-5eca-429a-a6f0-c0aface02695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:42.239869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50dd7a-996d-4008-a11d-82f26c60e794 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.260639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.307372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 812/1607] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:42.314920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cd50c-bed8-4519-a80d-ae6bd31ff362 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.316038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cd50c-bed8-4519-a80d-ae6bd31ff362 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.316780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cd50c-bed8-4519-a80d-ae6bd31ff362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.379720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d601e9-0525-4842-86bc-7f3a59dfe788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.380340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d601e9-0525-4842-86bc-7f3a59dfe788 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.380625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d601e9-0525-4842-86bc-7f3a59dfe788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.380898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d601e9-0525-4842-86bc-7f3a59dfe788 admin admin] PATH_INFO: `/v3/projects/6840c8d2676542ecbed0e9a237e489f5/users/9ea7096e5ac84095a0d6ff2b048afd91/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.398631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9c9cd50c-bed8-4519-a80d-ae6bd31ff362 None None] MFA Rules not processed for user `1ff51837b33e404d91fc3754c453e99c`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:42.400046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9c9cd50c-bed8-4519-a80d-ae6bd31ff362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:42.404045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d601e9-0525-4842-86bc-7f3a59dfe788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6840c8d2676542ecbed0e9a237e489f5, user_id=9ea7096e5ac84095a0d6ff2b048afd91, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.407069 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d601e9-0525-4842-86bc-7f3a59dfe788 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.411165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.447883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 796/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:42 2022] PUT /identity/v3/projects/6840c8d2676542ecbed0e9a237e489f5/users/9ea7096e5ac84095a0d6ff2b048afd91/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:42.451420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b304a-3d0c-419c-a471-b8af6635734f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.451911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b304a-3d0c-419c-a471-b8af6635734f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.452413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b304a-3d0c-419c-a471-b8af6635734f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.482983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f94b304a-3d0c-419c-a471-b8af6635734f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:42.484957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f94b304a-3d0c-419c-a471-b8af6635734f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:42.559016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 813/1609] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:42.561628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27b7c9-efe0-4f89-870b-c1340a562b92 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.562115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27b7c9-efe0-4f89-870b-c1340a562b92 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.562690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27b7c9-efe0-4f89-870b-c1340a562b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.599826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 797/1610] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:42.601530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a5912-7468-42f1-bf94-6706a65891c0 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.602053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a5912-7468-42f1-bf94-6706a65891c0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.602599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a5912-7468-42f1-bf94-6706a65891c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.611007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ae27b7c9-efe0-4f89-870b-c1340a562b92 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:42.614754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ae27b7c9-efe0-4f89-870b-c1340a562b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:42.636975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 814/1611] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:42.636975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27b7c9-efe0-4f89-870b-c1340a562b92 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.668269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-155a5912-7468-42f1-bf94-6706a65891c0 None None] MFA Rules not processed for user `0f4970620f8b4a9aa7eee97a89ef56b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:42.670783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-155a5912-7468-42f1-bf94-6706a65891c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:42.745474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27b7c9-efe0-4f89-870b-c1340a562b92 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.750952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.752429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.753032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.753459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.766812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.767881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.780378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 798/1612] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:42.781764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfa39-23a9-40ed-8698-250a87ff5960 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.783037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfa39-23a9-40ed-8698-250a87ff5960 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.783321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfa39-23a9-40ed-8698-250a87ff5960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.785736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 815/1613] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:42.786697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.798956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784db12-c2e4-43d8-9dc4-7de49137be24 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.805695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.806506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.807192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.807861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.808558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.811437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.834211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d99bfa39-23a9-40ed-8698-250a87ff5960 None None] MFA Rules not processed for user `9ea7096e5ac84095a0d6ff2b048afd91`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:42.836447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d99bfa39-23a9-40ed-8698-250a87ff5960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:42.861143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 816/1614] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:42.862687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.968455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 799/1615] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:42.970194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bfa39-23a9-40ed-8698-250a87ff5960 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.985808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bfa39-23a9-40ed-8698-250a87ff5960 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.000490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.008430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8523b-881e-40ac-8476-72dac6626d20 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.020639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d047cf5-32d0-4bac-8f96-6719dd7423c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.021426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d047cf5-32d0-4bac-8f96-6719dd7423c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.021808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d047cf5-32d0-4bac-8f96-6719dd7423c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.022467 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d047cf5-32d0-4bac-8f96-6719dd7423c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.049518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca674a-ece0-4520-acb0-c3eefb3233ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.050117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca674a-ece0-4520-acb0-c3eefb3233ae service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.050424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca674a-ece0-4520-acb0-c3eefb3233ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.050888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca674a-ece0-4520-acb0-c3eefb3233ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.052119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.134477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca674a-ece0-4520-acb0-c3eefb3233ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.136969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca674a-ece0-4520-acb0-c3eefb3233ae service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.150144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 800/1616] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:43.151709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc5a34-4a6c-4c6a-9017-039fa24ba71f None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.152004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc5a34-4a6c-4c6a-9017-039fa24ba71f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.152366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc5a34-4a6c-4c6a-9017-039fa24ba71f None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.159803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 801/1617] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 10 16:54:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:43.161023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc5a34-4a6c-4c6a-9017-039fa24ba71f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.177378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc5a34-4a6c-4c6a-9017-039fa24ba71f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.181577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d047cf5-32d0-4bac-8f96-6719dd7423c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.182875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c2e40-c64d-4e86-9d88-d4d5a2ecd1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.183271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d047cf5-32d0-4bac-8f96-6719dd7423c5 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.183813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c2e40-c64d-4e86-9d88-d4d5a2ecd1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.184104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c2e40-c64d-4e86-9d88-d4d5a2ecd1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.185223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c2e40-c64d-4e86-9d88-d4d5a2ecd1f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.185919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c2e40-c64d-4e86-9d88-d4d5a2ecd1f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.188136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c2e40-c64d-4e86-9d88-d4d5a2ecd1f0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.213228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 817/1618] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:43.215925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312060d0-9fbc-4187-8ca8-98c38ddfcae7 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.217364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312060d0-9fbc-4187-8ca8-98c38ddfcae7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.218354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312060d0-9fbc-4187-8ca8-98c38ddfcae7 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.223847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 818/1619] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 10 16:54:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:43.226905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 802/1620] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:43 2022] POST /identity/v3/users => generated 288 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:43.227649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29e9a8-37cd-4870-922a-a645812e9e41 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.227997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29e9a8-37cd-4870-922a-a645812e9e41 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.228415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29e9a8-37cd-4870-922a-a645812e9e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.230780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d137d-97a4-4e80-8f0b-2f1fad191fac None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.233132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d137d-97a4-4e80-8f0b-2f1fad191fac None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.233434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d137d-97a4-4e80-8f0b-2f1fad191fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.330723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-2a29e9a8-37cd-4870-922a-a645812e9e41 None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:43.331974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2a29e9a8-37cd-4870-922a-a645812e9e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:43.348192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1f8d137d-97a4-4e80-8f0b-2f1fad191fac None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:43.349468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1f8d137d-97a4-4e80-8f0b-2f1fad191fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:43.514744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 819/1621] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 10 16:54:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:43.516543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29e9a8-37cd-4870-922a-a645812e9e41 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.533613 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 803/1622] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 10 16:54:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:43.534958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d137d-97a4-4e80-8f0b-2f1fad191fac None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.543095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29e9a8-37cd-4870-922a-a645812e9e41 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.555915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.556750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.557031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.557548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.560471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.562070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d137d-97a4-4e80-8f0b-2f1fad191fac None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.564638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.576617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.577356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.577628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.577906 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.589431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 820/1623] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:43.590896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.631544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acf48d-fb89-4ff0-bb58-9c80853e113e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.650073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.650706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.650986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.651854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.651982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.729278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.732176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.769553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 804/1624] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:43.775405 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.810221 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8f4b3-1528-44e3-9c5f-a24a47971ce1 service nova] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.826922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.828680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.830212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.831269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.831558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.832457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] PATH_INFO: `/v3/projects/b334edf25de045aaa27a0397a013c8c2/users/1f10fa8e29bc4d88ad563b27f33b533d/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.853156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 821/1625] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:43.861214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b334edf25de045aaa27a0397a013c8c2, user_id=1f10fa8e29bc4d88ad563b27f33b533d, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.870213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.904447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 805/1626] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:43 2022] PUT /identity/v3/projects/b334edf25de045aaa27a0397a013c8c2/users/1f10fa8e29bc4d88ad563b27f33b533d/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:43.908189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.927227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bbc23-37c2-41e8-9687-b4a6c6939af9 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.956348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:44.108105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283c891-48e1-4130-951d-1016c3472176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.109007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283c891-48e1-4130-951d-1016c3472176 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.109457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283c891-48e1-4130-951d-1016c3472176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.109885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283c891-48e1-4130-951d-1016c3472176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.110694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283c891-48e1-4130-951d-1016c3472176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.112495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283c891-48e1-4130-951d-1016c3472176 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.140573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 822/1627] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:54:44.144980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.161066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43e609-de43-47f5-aa06-d3883a0b6efd admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.173270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709144e8-480d-425d-a772-f69ca772ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.173968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709144e8-480d-425d-a772-f69ca772ac3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.174251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709144e8-480d-425d-a772-f69ca772ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.176456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709144e8-480d-425d-a772-f69ca772ac3c admin admin] PATH_INFO: `/v3/projects/b334edf25de045aaa27a0397a013c8c2/users/1f10fa8e29bc4d88ad563b27f33b533d/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.181424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709144e8-480d-425d-a772-f69ca772ac3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b334edf25de045aaa27a0397a013c8c2, user_id=1f10fa8e29bc4d88ad563b27f33b533d, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.191292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709144e8-480d-425d-a772-f69ca772ac3c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.222188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 806/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:44 2022] PUT /identity/v3/projects/b334edf25de045aaa27a0397a013c8c2/users/1f10fa8e29bc4d88ad563b27f33b533d/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:44.232842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645e9a-2ef6-495f-9bd2-588c815bdc25 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.233360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645e9a-2ef6-495f-9bd2-588c815bdc25 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.233477 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645e9a-2ef6-495f-9bd2-588c815bdc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.322079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a7645e9a-2ef6-495f-9bd2-588c815bdc25 None None] MFA Rules not processed for user `1f10fa8e29bc4d88ad563b27f33b533d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:44.325471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a7645e9a-2ef6-495f-9bd2-588c815bdc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:44.366764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082927b-0d1f-4976-a5fc-50b5ecfb3b6a None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.367055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082927b-0d1f-4976-a5fc-50b5ecfb3b6a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.367329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082927b-0d1f-4976-a5fc-50b5ecfb3b6a None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.368811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 807/1629] 10.222.0.25 () {54 vars in 912 bytes} [Wed Aug 10 16:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:44.375035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c2929-4723-41f6-9aa1-ffaac2e20c20 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.375263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c2929-4723-41f6-9aa1-ffaac2e20c20 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.375533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c2929-4723-41f6-9aa1-ffaac2e20c20 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.377309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 808/1630] 10.222.0.25 () {54 vars in 912 bytes} [Wed Aug 10 16:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:44.378565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a16c8-1061-4176-977f-bef0815ccac1 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.378859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a16c8-1061-4176-977f-bef0815ccac1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.379139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a16c8-1061-4176-977f-bef0815ccac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.420716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-549a16c8-1061-4176-977f-bef0815ccac1 None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:44.421968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-549a16c8-1061-4176-977f-bef0815ccac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:44.459148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 823/1631] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:44.460916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1592ff-602c-43ab-bac4-5e66c0ec377f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.461401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1592ff-602c-43ab-bac4-5e66c0ec377f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.461852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1592ff-602c-43ab-bac4-5e66c0ec377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.486753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-cd1592ff-602c-43ab-bac4-5e66c0ec377f None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:44.488739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-cd1592ff-602c-43ab-bac4-5e66c0ec377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:44.588706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 809/1632] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:54:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:44.589517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a16c8-1061-4176-977f-bef0815ccac1 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.611671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 824/1633] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:54:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:44.613506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1592ff-602c-43ab-bac4-5e66c0ec377f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.655560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:44.698299 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1592ff-602c-43ab-bac4-5e66c0ec377f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.702582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.703282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.703646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.704283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.717396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.719402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.721782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a16c8-1061-4176-977f-bef0815ccac1 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.726038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.727901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.727901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.727901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.736486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 825/1634] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:44.743138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.746001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.769909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 810/1635] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:44.894638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.912670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d6fed-8871-4962-8a3d-3f9f4265d664 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.922258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43adbb-3d62-4b25-9dcb-5873b1751aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.923953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43adbb-3d62-4b25-9dcb-5873b1751aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.924824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43adbb-3d62-4b25-9dcb-5873b1751aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.925976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43adbb-3d62-4b25-9dcb-5873b1751aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.953054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43adbb-3d62-4b25-9dcb-5873b1751aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.954519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43adbb-3d62-4b25-9dcb-5873b1751aa7 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.986841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 826/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:45.331420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:45.358302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:45.477820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385741b-9db3-4ea7-bc1e-699406be9458 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:45.492306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:45.493214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.493596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.494115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] PATH_INFO: `/v3/users/0f4970620f8b4a9aa7eee97a89ef56b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.497407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4970620f8b4a9aa7eee97a89ef56b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:45.504560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:45.645570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4970620f8b4a9aa7eee97a89ef56b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:45.670896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4970620f8b4a9aa7eee97a89ef56b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:45.700549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4970620f8b4a9aa7eee97a89ef56b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:45.700549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4970620f8b4a9aa7eee97a89ef56b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:45.733709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4970620f8b4a9aa7eee97a89ef56b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:45.746274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 811/1637] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:54:45 2022] DELETE /identity/v3/users/0f4970620f8b4a9aa7eee97a89ef56b7 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:45.748616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.749078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.750697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.788572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:45.813389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:45.979881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:45.982990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:46.040768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 827/1638] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:46.095062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.101231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:46.273377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642c226-4a40-4bd1-8aeb-c649d0132ce1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.275711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e9ea4-306c-42b6-867b-4d33718d2df0 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.282336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c8951-1cd7-4af5-8922-c4a598e78e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.284553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c8951-1cd7-4af5-8922-c4a598e78e91 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.285054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c8951-1cd7-4af5-8922-c4a598e78e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.285824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c8951-1cd7-4af5-8922-c4a598e78e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.295124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.299794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.301784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.302355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] PATH_INFO: `/v3/users/1ff51837b33e404d91fc3754c453e99c` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.310353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff51837b33e404d91fc3754c453e99c)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:46.315952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:46.335357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c8951-1cd7-4af5-8922-c4a598e78e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:46.339193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c8951-1cd7-4af5-8922-c4a598e78e91 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:46.365825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 828/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:54:46.447910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:46.459586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff51837b33e404d91fc3754c453e99c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:46.480025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff51837b33e404d91fc3754c453e99c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:46.506249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff51837b33e404d91fc3754c453e99c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:46.507151 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff51837b33e404d91fc3754c453e99c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:46.542283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff51837b33e404d91fc3754c453e99c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:46.556198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 812/1640] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:54:45 2022] DELETE /identity/v3/users/1ff51837b33e404d91fc3754c453e99c => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:46.561264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8e346-a979-4429-a758-7ca503375719 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.561881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8e346-a979-4429-a758-7ca503375719 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.562318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8e346-a979-4429-a758-7ca503375719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.597323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-16d8e346-a979-4429-a758-7ca503375719 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:46.598766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-16d8e346-a979-4429-a758-7ca503375719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:46.738339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:46.757307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 829/1641] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:46.816739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.818253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:46.975166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc15ef-fbe8-45c9-9955-e226d1300f6e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.989962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e15942-faae-4be6-822f-68897cd691cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.990603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15942-faae-4be6-822f-68897cd691cb service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.990885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15942-faae-4be6-822f-68897cd691cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.991244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15942-faae-4be6-822f-68897cd691cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.022837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e15942-faae-4be6-822f-68897cd691cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:47.024231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e15942-faae-4be6-822f-68897cd691cb service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:47.051419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 813/1642] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:48.845474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8e346-a979-4429-a758-7ca503375719 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:48.846951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-16d8e346-a979-4429-a758-7ca503375719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:48.871954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8e346-a979-4429-a758-7ca503375719 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.885949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.887999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.889679 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.890738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] PATH_INFO: `/v3/projects/333933f84263401dbcfee37d9078aba0` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.901493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=333933f84263401dbcfee37d9078aba0)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:48.903562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.039157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333933f84263401dbcfee37d9078aba0', 'request_id': 'req-26d9b70b-68f1-4011-b725-a9da58848277'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.061205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333933f84263401dbcfee37d9078aba0', 'request_id': 'req-26d9b70b-68f1-4011-b725-a9da58848277'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.070764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333933f84263401dbcfee37d9078aba0', 'request_id': 'req-26d9b70b-68f1-4011-b725-a9da58848277'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.107141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] The token cache is being invalidate because project 333933f84263401dbcfee37d9078aba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:54:49.107404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.108075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.113330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 830/1643] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:54:48 2022] DELETE /identity/v3/projects/333933f84263401dbcfee37d9078aba0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:49.115163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0890e-5644-48b8-ba70-00c35c4df826 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.115965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0890e-5644-48b8-ba70-00c35c4df826 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.116409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0890e-5644-48b8-ba70-00c35c4df826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.157590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-33f0890e-5644-48b8-ba70-00c35c4df826 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:49.158791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-33f0890e-5644-48b8-ba70-00c35c4df826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:49.165333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.166913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:49.361074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9b70b-68f1-4011-b725-a9da58848277 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.367390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 814/1644] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:49.369454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.370332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.370782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.371200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] PATH_INFO: `/v3/projects/f7d47026aa0a4257a64b7b1ece2a574d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.377109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7400ecf-29cb-4fba-8e54-de9b7d198021 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.377676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7400ecf-29cb-4fba-8e54-de9b7d198021 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.378093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7400ecf-29cb-4fba-8e54-de9b7d198021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.378893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d47026aa0a4257a64b7b1ece2a574d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.386790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.421607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a7400ecf-29cb-4fba-8e54-de9b7d198021 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:49.425067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a7400ecf-29cb-4fba-8e54-de9b7d198021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:49.466227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 815/1645] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:49.467438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7400ecf-29cb-4fba-8e54-de9b7d198021 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.516391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7400ecf-29cb-4fba-8e54-de9b7d198021 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.540729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d47026aa0a4257a64b7b1ece2a574d', 'request_id': 'req-32de2cb0-c9e4-4a80-832b-3daa87edac00'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.560797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d47026aa0a4257a64b7b1ece2a574d', 'request_id': 'req-32de2cb0-c9e4-4a80-832b-3daa87edac00'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.577218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d47026aa0a4257a64b7b1ece2a574d', 'request_id': 'req-32de2cb0-c9e4-4a80-832b-3daa87edac00'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.610757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] The token cache is being invalidate because project f7d47026aa0a4257a64b7b1ece2a574d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:54:49.611121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.612485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:49.616534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 831/1646] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:54:49 2022] DELETE /identity/v3/projects/f7d47026aa0a4257a64b7b1ece2a574d => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:49.620360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.625628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:49.675504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:49.676467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:49.710726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.710726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.711033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.711735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.711836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:49.712178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.717450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.762921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 816/1647] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:49.766522 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.787971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:49.839115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fbaf3-2c7c-4052-b24f-12ed457d6a2e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.864092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.867690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.868063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.868335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.868985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.877241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.888813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de2cb0-c9e4-4a80-832b-3daa87edac00 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.901486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.902243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.902763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.902917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] PATH_INFO: `/v3/users/1f10fa8e29bc4d88ad563b27f33b533d` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.906124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f10fa8e29bc4d88ad563b27f33b533d)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.908650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: WARNING py.warnings [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:54:49.908650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: warnings.warn(msg) Aug 10 16:54:49.908650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: Aug 10 16:54:49.911440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.963594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 817/1648] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:49 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:49.966373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.091454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f10fa8e29bc4d88ad563b27f33b533d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:50.092768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f10fa8e29bc4d88ad563b27f33b533d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:50.117413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f10fa8e29bc4d88ad563b27f33b533d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:50.118157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa27d9-270b-47e7-9332-3fd7b88a0e86 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.119361 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f10fa8e29bc4d88ad563b27f33b533d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:50.125643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.126449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.126823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.128083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.129061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.131535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.155989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f10fa8e29bc4d88ad563b27f33b533d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:50.181014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 818/1649] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:49 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:50.186851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.191627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:50.211649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 832/1650] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:54:49 2022] DELETE /identity/v3/users/1f10fa8e29bc4d88ad563b27f33b533d => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:50.220093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f2226-f1c3-4637-8e3a-62931e04898f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.308252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.310102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.310102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.310361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.310875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.312555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.329798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 819/1651] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:50.332847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.349387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aabace-273e-44e7-b4e2-d404fea61ed5 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.357240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825ad88-7b79-4ff1-8105-f53f79e910dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.358034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ad88-7b79-4ff1-8105-f53f79e910dc admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.358430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ad88-7b79-4ff1-8105-f53f79e910dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.358836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ad88-7b79-4ff1-8105-f53f79e910dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.359621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825ad88-7b79-4ff1-8105-f53f79e910dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.361738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825ad88-7b79-4ff1-8105-f53f79e910dc admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.390766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 833/1652] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:50.393545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.410761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf92c24-11de-409c-b947-7165d3e12a3c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.419664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.420410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.423744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.423744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] PATH_INFO: `/v3/projects/ad8022c7f36c452bab1b19a60b0c2f8a/users/b31791cb6fe74fb89f5258af76998d85/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.487352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8022c7f36c452bab1b19a60b0c2f8a, user_id=b31791cb6fe74fb89f5258af76998d85, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.495213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.543011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 820/1653] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:50 2022] PUT /identity/v3/projects/ad8022c7f36c452bab1b19a60b0c2f8a/users/b31791cb6fe74fb89f5258af76998d85/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:50.546213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20e908-0e0c-43f9-b615-0ba6df8cfbf1 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.546533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20e908-0e0c-43f9-b615-0ba6df8cfbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.546823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20e908-0e0c-43f9-b615-0ba6df8cfbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.624088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4f20e908-0e0c-43f9-b615-0ba6df8cfbf1 None None] MFA Rules not processed for user `b31791cb6fe74fb89f5258af76998d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:50.625924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4f20e908-0e0c-43f9-b615-0ba6df8cfbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:50.852838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 834/1654] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:50.858092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.940186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.006458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e38d6-d41e-47a7-a472-70495d5b15e5 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.015582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.016626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.016626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.016835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.017488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.019606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.076088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 821/1655] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:50 2022] POST /identity/v3/projects => generated 365 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:54:51.077263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20e908-0e0c-43f9-b615-0ba6df8cfbf1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.095954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20e908-0e0c-43f9-b615-0ba6df8cfbf1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.132522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.240875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.244602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.244834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.245116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.245793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.258581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.294700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 835/1656] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:51 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:54:51.298573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.370511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7bc11-3a31-4f1a-badd-d6cf353a5b6c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.386756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.391108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.391611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.392424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.393822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.401779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.435235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 822/1657] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:54:51.439657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.461570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143eedad-864a-4718-8148-863a108ae740 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.479933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.480682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.487058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.487690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] PATH_INFO: `/v3/projects/d422a40e978e4159823d860e0e1ac6a3/users/b770b4b39f6d4fefb843ce2248f6adc8/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.540795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d422a40e978e4159823d860e0e1ac6a3, user_id=b770b4b39f6d4fefb843ce2248f6adc8, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.544031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.582359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 836/1658] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:51 2022] PUT /identity/v3/projects/d422a40e978e4159823d860e0e1ac6a3/users/b770b4b39f6d4fefb843ce2248f6adc8/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:51.586971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.644775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f61e4-4d9b-4581-805d-7d311d1ad33b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.792072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fa6ed-d7e7-4f1d-bc34-f3eff2662957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.793076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fa6ed-d7e7-4f1d-bc34-f3eff2662957 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.795154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fa6ed-d7e7-4f1d-bc34-f3eff2662957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.796605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fa6ed-d7e7-4f1d-bc34-f3eff2662957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.797482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fa6ed-d7e7-4f1d-bc34-f3eff2662957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.800524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fa6ed-d7e7-4f1d-bc34-f3eff2662957 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.812676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 823/1659] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:51.817193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.833931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497148f5-3b8e-4fbe-8ebb-3ed686ff455d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.850434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.850824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.851106 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.851382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] PATH_INFO: `/v3/projects/d422a40e978e4159823d860e0e1ac6a3/users/b770b4b39f6d4fefb843ce2248f6adc8/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.863088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d422a40e978e4159823d860e0e1ac6a3, user_id=b770b4b39f6d4fefb843ce2248f6adc8, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.881413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.933243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 837/1660] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:51 2022] PUT /identity/v3/projects/d422a40e978e4159823d860e0e1ac6a3/users/b770b4b39f6d4fefb843ce2248f6adc8/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:51.946912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.947428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.947961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.036127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] MFA Rules not processed for user `b770b4b39f6d4fefb843ce2248f6adc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:52.037414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:52.213106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 824/1661] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:52.224424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:52.227372 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:52.370923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c169064-9214-45a1-bdaa-760ccfc1f43c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:52.388947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.390236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.390660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.391082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.405232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.407208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.430277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 838/1662] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:52.467841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:52.469795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:52.572238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:52.576956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:52.597210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c405723-c8a3-457b-8f0e-67f42b64ae89 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:52.604127 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3583abc-6285-4e14-b46b-ffae9841c52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.605496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3583abc-6285-4e14-b46b-ffae9841c52d service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.606731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3583abc-6285-4e14-b46b-ffae9841c52d service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.608073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3583abc-6285-4e14-b46b-ffae9841c52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.685142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd1bd-6122-49cf-937b-5c6ff0c38e6b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:52.695637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.698459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.698601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.698973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] PATH_INFO: `/v3/projects/b334edf25de045aaa27a0397a013c8c2` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.702088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b334edf25de045aaa27a0397a013c8c2)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.703948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: WARNING py.warnings [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 16:54:52.703948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: warnings.warn(msg) Aug 10 16:54:52.703948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: Aug 10 16:54:52.705527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.771895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3583abc-6285-4e14-b46b-ffae9841c52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.773777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3583abc-6285-4e14-b46b-ffae9841c52d service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.797625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 839/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:54:52.872413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b334edf25de045aaa27a0397a013c8c2', 'request_id': 'req-e51f55ee-e340-4f9d-8975-21bcdf53cc82'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:52.892190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b334edf25de045aaa27a0397a013c8c2', 'request_id': 'req-e51f55ee-e340-4f9d-8975-21bcdf53cc82'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:52.913936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b334edf25de045aaa27a0397a013c8c2', 'request_id': 'req-e51f55ee-e340-4f9d-8975-21bcdf53cc82'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:52.962887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] The token cache is being invalidate because project b334edf25de045aaa27a0397a013c8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:54:52.963435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:52.965116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e51f55ee-e340-4f9d-8975-21bcdf53cc82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:52.971880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 825/1664] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:54:52 2022] DELETE /identity/v3/projects/b334edf25de045aaa27a0397a013c8c2 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:52.973419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e01a4-8c05-4f52-b14e-d1b78953f222 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.973748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e01a4-8c05-4f52-b14e-d1b78953f222 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.974029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e01a4-8c05-4f52-b14e-d1b78953f222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.995718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:53.068975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-993e01a4-8c05-4f52-b14e-d1b78953f222 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:53.070923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-993e01a4-8c05-4f52-b14e-d1b78953f222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:53.206186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 840/1665] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:54:53.209546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf33f-08cd-4354-a4e0-67ff065f13ce None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.210111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf33f-08cd-4354-a4e0-67ff065f13ce None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.210745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf33f-08cd-4354-a4e0-67ff065f13ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.239927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9a3cf33f-08cd-4354-a4e0-67ff065f13ce None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:53.241704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9a3cf33f-08cd-4354-a4e0-67ff065f13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:53.256207 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 826/1666] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:53.258684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e01a4-8c05-4f52-b14e-d1b78953f222 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:53.273341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e01a4-8c05-4f52-b14e-d1b78953f222 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:53.279659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:53.280582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.280947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.281292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.282035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:53.282615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:53.285792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:53.304551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 841/1667] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:53.306065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf33f-08cd-4354-a4e0-67ff065f13ce None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:53.319532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf33f-08cd-4354-a4e0-67ff065f13ce None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:53.326896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace1968-76fc-40ad-a1c6-5181da72210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:53.327553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace1968-76fc-40ad-a1c6-5181da72210b admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.327840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace1968-76fc-40ad-a1c6-5181da72210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.328335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace1968-76fc-40ad-a1c6-5181da72210b admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.328982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace1968-76fc-40ad-a1c6-5181da72210b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:53.331081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace1968-76fc-40ad-a1c6-5181da72210b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:53.394894 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 827/1668] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:53 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:53.399300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:53.411632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77bd40-40b5-41ad-a96b-4617aaf1ff86 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:53.593147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:53.594801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.595102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.595389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.597673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:53.600116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:53.661690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 842/1669] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:53 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:53.663358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2889c-4099-46a4-998b-145c37063d16 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.663809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2889c-4099-46a4-998b-145c37063d16 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.663949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2889c-4099-46a4-998b-145c37063d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.666298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:53.689138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04873b-182f-44e6-bc17-6b3e167ff4ad admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:53.706414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:53.707050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.709567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.709958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.710647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:53.718470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:53.754218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 843/1670] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:54:53.759015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b8d2889c-4099-46a4-998b-145c37063d16 None None] MFA Rules not processed for user `6677b89e26a94e5da074431e11e12050`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:53.760548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b8d2889c-4099-46a4-998b-145c37063d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:53.764685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:53.785816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bd006-5604-4609-832e-8280e21388c3 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:53.800983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:53.801802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.801927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.802153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.802832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:53.804969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:53.839292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 844/1671] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:54:53.843438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:53.871669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6629cd1-1f17-41d7-af4f-06537f3023a7 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:53.900924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67505a-a555-488a-b374-3b40ce1f90b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:53.901718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67505a-a555-488a-b374-3b40ce1f90b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.902144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67505a-a555-488a-b374-3b40ce1f90b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.902230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67505a-a555-488a-b374-3b40ce1f90b9 admin admin] PATH_INFO: `/v3/projects/bc4c056435b4445381f39bb95120611e/users/1f40a2a4a4554cf9a901c27c7c6d5d82/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.971582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 828/1672] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:53.973722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:53.974285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:53.975419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:53.987647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67505a-a555-488a-b374-3b40ce1f90b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4c056435b4445381f39bb95120611e, user_id=1f40a2a4a4554cf9a901c27c7c6d5d82, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:53.991184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67505a-a555-488a-b374-3b40ce1f90b9 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:54.009382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 845/1673] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:53 2022] PUT /identity/v3/projects/bc4c056435b4445381f39bb95120611e/users/1f40a2a4a4554cf9a901c27c7c6d5d82/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:54.009804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] MFA Rules not processed for user `6677b89e26a94e5da074431e11e12050`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:54.012030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:54.022721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d057-b80a-4648-b595-0eada01bced7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:54.026794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d057-b80a-4648-b595-0eada01bced7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:54.028234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d057-b80a-4648-b595-0eada01bced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:54.033951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:54.119726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 829/1674] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:54.145257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5e48d057-b80a-4648-b595-0eada01bced7 None None] MFA Rules not processed for user `1f40a2a4a4554cf9a901c27c7c6d5d82`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:54.146550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5e48d057-b80a-4648-b595-0eada01bced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:54.154622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:54.158900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:54.209697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:54.314103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 846/1675] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:54.318251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48d057-b80a-4648-b595-0eada01bced7 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:54.381390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f284269-5a5f-4003-88ab-c5a7710f2bc2 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:54.395713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:54.396701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:54.396979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:54.397252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:54.417650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:54.422438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:54.450000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 830/1676] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:54.481746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48d057-b80a-4648-b595-0eada01bced7 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:54.498264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:54.498916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:54.499203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:54.499470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:54.500111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:54.502096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:54.582947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 847/1677] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:54 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:54:54.588741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:54.623699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9315-affb-4acf-ae4a-72b97ab6b1c0 service nova] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:54.722926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:54.724442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:54.724974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:54.725498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:54.726386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:54.728832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:54.765434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 831/1678] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:54:54 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:54.765594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:54.779544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc6b77-69e5-492d-8bca-a4323d0536d0 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:54.786358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:54.787295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:54.787769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:54.788319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:54.789190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:54.791938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:54.810656 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:54.821743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 848/1679] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:54:54.823615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f223b3-1e5e-4f12-bbea-aceadb20fdb3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:54.830271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:54.831084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:54.831463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:54.831994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] PATH_INFO: `/v3/projects/fc2b850bf10549bfac5d1fe69b3e2b49/users/87a27812c5af4722ba37fd423824e9df/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:54.866471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2b850bf10549bfac5d1fe69b3e2b49, user_id=87a27812c5af4722ba37fd423824e9df, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:54.870590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:54.894792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:54.895167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 832/1680] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:54 2022] PUT /identity/v3/projects/fc2b850bf10549bfac5d1fe69b3e2b49/users/87a27812c5af4722ba37fd423824e9df/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:54.906670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cdc29-7963-4114-978c-b570a11f7cb4 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:55.040545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c6169-a9c0-4574-8401-07a688ee5548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:55.041328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c6169-a9c0-4574-8401-07a688ee5548 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:55.041675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c6169-a9c0-4574-8401-07a688ee5548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:55.047745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c6169-a9c0-4574-8401-07a688ee5548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:55.048533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c6169-a9c0-4574-8401-07a688ee5548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:55.050403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c6169-a9c0-4574-8401-07a688ee5548 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:55.080492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:55.087224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 849/1681] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:54:55.094916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda256-df22-49e4-bcb0-d359ee133c7c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:55.106325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:55.108093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:55.108674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:55.109320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] PATH_INFO: `/v3/projects/fc2b850bf10549bfac5d1fe69b3e2b49/users/87a27812c5af4722ba37fd423824e9df/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:55.114095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2b850bf10549bfac5d1fe69b3e2b49, user_id=87a27812c5af4722ba37fd423824e9df, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:55.120104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:55.140794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:55.152983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 833/1682] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:54:55 2022] PUT /identity/v3/projects/fc2b850bf10549bfac5d1fe69b3e2b49/users/87a27812c5af4722ba37fd423824e9df/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:55.164133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b73b-7622-4000-bb14-be482828bf70 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:55.164383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b73b-7622-4000-bb14-be482828bf70 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:55.164801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b73b-7622-4000-bb14-be482828bf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:55.224309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9332b73b-7622-4000-bb14-be482828bf70 None None] MFA Rules not processed for user `87a27812c5af4722ba37fd423824e9df`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:55.228067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9332b73b-7622-4000-bb14-be482828bf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:55.298937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:55.430410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 850/1683] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:55.444348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:55.446182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:55.574283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ed513-efd4-4724-a73f-20602ac4158e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:55.586043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4714954-baf5-4fdf-9e27-a714058666fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:55.587032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714954-baf5-4fdf-9e27-a714058666fc service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:55.587580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714954-baf5-4fdf-9e27-a714058666fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:55.587580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714954-baf5-4fdf-9e27-a714058666fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:55.610193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4714954-baf5-4fdf-9e27-a714058666fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:55.611889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4714954-baf5-4fdf-9e27-a714058666fc service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:55.634212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 834/1684] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:56.156743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.157199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.157631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.236523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167fa71-ca57-49fa-abd2-4ec822d7b0d1 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.236990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167fa71-ca57-49fa-abd2-4ec822d7b0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.237095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167fa71-ca57-49fa-abd2-4ec822d7b0d1 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.238783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 835/1685] 10.222.0.25 () {54 vars in 912 bytes} [Wed Aug 10 16:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:56.241153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] MFA Rules not processed for user `7a701149974b45db9f782bc5a232aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:56.242389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:56.248986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.249275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.249549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.405215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] MFA Rules not processed for user `256af8436dd842fdb4ebe90ec9be26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:56.411464 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:56.479236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 851/1686] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:56.488743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.490319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:56.566157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 836/1687] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:54:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:54:56.644242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.645856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:56.663278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb01016-ea2b-44f1-b6b0-854abccb5cc8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:56.674593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:56.675247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.675534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.678580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.692035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14597591-8fe9-4cc7-b426-295958f8c901 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:56.698728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:56.705006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:56.713278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:56.716788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.717270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.719095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.731973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 837/1688] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:54:56.737092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.759758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ace90-2f51-4a46-8430-35d99880c9d4 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:56.771535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:56.773960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87a27812c5af4722ba37fd423824e9df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc2b850bf10549bfac5d1fe69b3e2b49', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:56.774614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.774931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.775183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.775848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:56.776187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:56.777028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:56.778344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:56.808271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 852/1689] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:54:56.825389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 838/1690] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:56.829660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.849589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6241b-363c-41c7-8026-0fc7a7868706 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:56.858539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87a27812c5af4722ba37fd423824e9df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc2b850bf10549bfac5d1fe69b3e2b49', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:56.859394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.859818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.860297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.861091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:56.863513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:56.931303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 853/1691] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:54:56 2022] POST /identity/v3/projects => generated 355 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:56.935296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.949984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb33bf-10e1-4dc6-89cb-2dd14f63b565 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.100932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d8788-c307-44b6-a613-8a5aa1665a6f tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87a27812c5af4722ba37fd423824e9df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc2b850bf10549bfac5d1fe69b3e2b49', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.101919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8788-c307-44b6-a613-8a5aa1665a6f tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.102206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8788-c307-44b6-a613-8a5aa1665a6f tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.102574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8788-c307-44b6-a613-8a5aa1665a6f tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.103856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d8788-c307-44b6-a613-8a5aa1665a6f tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:57.105248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d8788-c307-44b6-a613-8a5aa1665a6f tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.108860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d8788-c307-44b6-a613-8a5aa1665a6f tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.140127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 839/1692] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:57.229591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.245397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8604-186c-4a39-bfb4-94f11b9137d3 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.255797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87a27812c5af4722ba37fd423824e9df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc2b850bf10549bfac5d1fe69b3e2b49', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.256531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.256819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.257094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] PATH_INFO: `/v3/projects/d36e890a65444d6d878f6b41ccc4bf62` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.260654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d36e890a65444d6d878f6b41ccc4bf62)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.265344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.293297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.420980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36e890a65444d6d878f6b41ccc4bf62', 'request_id': 'req-b5ee8779-0bd1-44b1-9525-b0498aa72c85'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:57.446466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36e890a65444d6d878f6b41ccc4bf62', 'request_id': 'req-b5ee8779-0bd1-44b1-9525-b0498aa72c85'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:57.461113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36e890a65444d6d878f6b41ccc4bf62', 'request_id': 'req-b5ee8779-0bd1-44b1-9525-b0498aa72c85'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:57.495995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] The token cache is being invalidate because project d36e890a65444d6d878f6b41ccc4bf62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:54:57.497705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:57.498134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b5ee8779-0bd1-44b1-9525-b0498aa72c85 tempest-VolumeQuotaClassesTest-1398355698 tempest-VolumeQuotaClassesTest-1398355698-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:57.500411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 854/1693] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:54:57 2022] DELETE /identity/v3/projects/d36e890a65444d6d878f6b41ccc4bf62 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:57.580965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.583453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.583763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.650303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.699982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] MFA Rules not processed for user `c7fac765624c4e0b85b01f356a14a1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:57.706628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:57.953208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 840/1694] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:57.962836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.963195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.963479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.979106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.984822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:58.064064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] MFA Rules not processed for user `c7fac765624c4e0b85b01f356a14a1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:58.066471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:58.095110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 855/1695] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:58.102699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:58.113948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:58.298768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220c647-6324-4c45-a3aa-613d5389e9ba None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:58.309671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:58.310344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.310627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.310904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] PATH_INFO: `/v3/users/b31791cb6fe74fb89f5258af76998d85` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.314589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31791cb6fe74fb89f5258af76998d85)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:58.320059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:58.348402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67918d-e3a4-459e-b62c-7e612454aa1f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:58.370433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:58.371294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.375550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.376082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.413337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:58.417579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:58.460717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 856/1696] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:58.461828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:58.466197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:58.487986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31791cb6fe74fb89f5258af76998d85'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.506101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05371-fbae-4dd9-a6db-ee7b32b6f3aa service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:58.509544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31791cb6fe74fb89f5258af76998d85'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.526051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31791cb6fe74fb89f5258af76998d85'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.526701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31791cb6fe74fb89f5258af76998d85'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.527049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:58.528482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.528927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.529290 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] PATH_INFO: `/v3/users/1f40a2a4a4554cf9a901c27c7c6d5d82` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.532594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f40a2a4a4554cf9a901c27c7c6d5d82)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:58.534693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:58.560019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9e498bcf-8e26-4e62-88aa-4c27f7f5eeac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31791cb6fe74fb89f5258af76998d85'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.577229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 841/1697] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:54:57 2022] DELETE /identity/v3/users/b31791cb6fe74fb89f5258af76998d85 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:58.584985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf770b1-ff57-4cc2-a80b-3704bed89339 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.585511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf770b1-ff57-4cc2-a80b-3704bed89339 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.585613 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf770b1-ff57-4cc2-a80b-3704bed89339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.631871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-eaf770b1-ff57-4cc2-a80b-3704bed89339 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:58.633548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-eaf770b1-ff57-4cc2-a80b-3704bed89339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:58.664643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:58.697516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f40a2a4a4554cf9a901c27c7c6d5d82'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.716450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f40a2a4a4554cf9a901c27c7c6d5d82'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.743468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f40a2a4a4554cf9a901c27c7c6d5d82'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.754292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f40a2a4a4554cf9a901c27c7c6d5d82'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.779476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:58.784768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f40a2a4a4554cf9a901c27c7c6d5d82'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:54:58.844438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 842/1698] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:58.850591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 857/1699] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:54:58 2022] DELETE /identity/v3/users/1f40a2a4a4554cf9a901c27c7c6d5d82 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:58.855153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcf33-b419-4c5c-b4f6-1e64e153cd85 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.855448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcf33-b419-4c5c-b4f6-1e64e153cd85 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.856399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcf33-b419-4c5c-b4f6-1e64e153cd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.915218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8ecfcf33-b419-4c5c-b4f6-1e64e153cd85 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:58.916512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8ecfcf33-b419-4c5c-b4f6-1e64e153cd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:58.919998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:58.924940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:59.163337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef7cf4-095a-479a-83fe-6636d2440864 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:59.173065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 843/1700] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:59.183277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598719f-4479-403b-8f2c-e93039007641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:59.184561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598719f-4479-403b-8f2c-e93039007641 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.185016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598719f-4479-403b-8f2c-e93039007641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.186130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598719f-4479-403b-8f2c-e93039007641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.213194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598719f-4479-403b-8f2c-e93039007641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:59.214737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598719f-4479-403b-8f2c-e93039007641 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:59.247840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 858/1701] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:54:59.299503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfcf33-b419-4c5c-b4f6-1e64e153cd85 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:59.312602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfcf33-b419-4c5c-b4f6-1e64e153cd85 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:59.322454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442fb87-c89e-4376-b730-f43ba0216654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:59.323665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442fb87-c89e-4376-b730-f43ba0216654 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.326870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442fb87-c89e-4376-b730-f43ba0216654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.327161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442fb87-c89e-4376-b730-f43ba0216654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.373229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442fb87-c89e-4376-b730-f43ba0216654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:59.380910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442fb87-c89e-4376-b730-f43ba0216654 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:59.444022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 844/1702] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:54:59.583706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d4eb0-44dc-4751-b880-df49e26d21c5 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.584276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d4eb0-44dc-4751-b880-df49e26d21c5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.584382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d4eb0-44dc-4751-b880-df49e26d21c5 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.587029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 859/1703] 10.222.0.25 () {54 vars in 961 bytes} [Wed Aug 10 16:54:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:59.595857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902ebff-c0ef-477d-9436-31faeddda606 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.596775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902ebff-c0ef-477d-9436-31faeddda606 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.597474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902ebff-c0ef-477d-9436-31faeddda606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.629747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d902ebff-c0ef-477d-9436-31faeddda606 None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:59.631749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d902ebff-c0ef-477d-9436-31faeddda606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:59.822119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d4eb0-44dc-4751-b880-df49e26d21c5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:59.823330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 845/1704] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Aug 10 16:54:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:54:59.846179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d4eb0-44dc-4751-b880-df49e26d21c5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:59.862569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:59.863169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.863447 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.863736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.865653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:59.904140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:59.906310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:59.935563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 860/1705] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:54:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:00.847037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.847600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.847695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.915387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] MFA Rules not processed for user `96495ad2ad764ba3a0fb1e3bc28fd8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:00.917649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:00.983881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:00.987335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:00.988646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.032619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a54963-5fc2-4d9e-af44-7d77b4b358eb service placement] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.041480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.043478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.043929 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.044600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] PATH_INFO: `/v3/projects/ad8022c7f36c452bab1b19a60b0c2f8a` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.048049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8022c7f36c452bab1b19a60b0c2f8a)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.050802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.136837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8022c7f36c452bab1b19a60b0c2f8a', 'request_id': 'req-9db710bb-b8d9-46e1-8f51-f0b6b735d336'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.145060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8022c7f36c452bab1b19a60b0c2f8a', 'request_id': 'req-9db710bb-b8d9-46e1-8f51-f0b6b735d336'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.153629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8022c7f36c452bab1b19a60b0c2f8a', 'request_id': 'req-9db710bb-b8d9-46e1-8f51-f0b6b735d336'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.162734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 846/1706] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:01.201158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] The token cache is being invalidate because project ad8022c7f36c452bab1b19a60b0c2f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:01.202382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.204070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.208790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.210516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:01.212875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 861/1707] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:00 2022] DELETE /identity/v3/projects/ad8022c7f36c452bab1b19a60b0c2f8a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:01.217316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.219786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:01.257635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeaa1e6-1323-4efd-aa03-25d6a2826814 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.376746 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db710bb-b8d9-46e1-8f51-f0b6b735d336 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.386291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.387047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.387293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.389764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] PATH_INFO: `/v3/users/b770b4b39f6d4fefb843ce2248f6adc8` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.391979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b770b4b39f6d4fefb843ce2248f6adc8)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.397962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.400288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.455784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb73278-da28-4875-b440-111782271c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.456655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb73278-da28-4875-b440-111782271c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.456938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb73278-da28-4875-b440-111782271c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.457784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb73278-da28-4875-b440-111782271c70 admin admin] PATH_INFO: `/v3/projects/bc4c056435b4445381f39bb95120611e` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.459718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb73278-da28-4875-b440-111782271c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4c056435b4445381f39bb95120611e)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.466055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb73278-da28-4875-b440-111782271c70 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.518598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b770b4b39f6d4fefb843ce2248f6adc8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.519663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b770b4b39f6d4fefb843ce2248f6adc8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.537810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b770b4b39f6d4fefb843ce2248f6adc8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.539135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b770b4b39f6d4fefb843ce2248f6adc8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.587893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b770b4b39f6d4fefb843ce2248f6adc8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.593406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-beb73278-da28-4875-b440-111782271c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4c056435b4445381f39bb95120611e', 'request_id': 'req-beb73278-da28-4875-b440-111782271c70'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.611738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-beb73278-da28-4875-b440-111782271c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4c056435b4445381f39bb95120611e', 'request_id': 'req-beb73278-da28-4875-b440-111782271c70'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.622645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 862/1708] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:01 2022] DELETE /identity/v3/users/b770b4b39f6d4fefb843ce2248f6adc8 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:01.623584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.624992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:01.644461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-beb73278-da28-4875-b440-111782271c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4c056435b4445381f39bb95120611e', 'request_id': 'req-beb73278-da28-4875-b440-111782271c70'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.680024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-beb73278-da28-4875-b440-111782271c70 admin admin] The token cache is being invalidate because project bc4c056435b4445381f39bb95120611e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:01.680619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-beb73278-da28-4875-b440-111782271c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.682502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-beb73278-da28-4875-b440-111782271c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:01.687891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 847/1709] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:01 2022] DELETE /identity/v3/projects/bc4c056435b4445381f39bb95120611e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:55:01.688368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704a224-2534-4c32-b78f-e81422cb0d6c None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.688762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704a224-2534-4c32-b78f-e81422cb0d6c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.689999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704a224-2534-4c32-b78f-e81422cb0d6c None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.694468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 848/1710] 10.222.0.25 () {54 vars in 912 bytes} [Wed Aug 10 16:55:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:01.695485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704a224-2534-4c32-b78f-e81422cb0d6c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.700768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5704a224-2534-4c32-b78f-e81422cb0d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:01.797995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4745ec-9b8f-4de0-b662-a8b8e55e0b24 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.815377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639d253-6425-4870-bd0d-b8dbdd1cf196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.816030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639d253-6425-4870-bd0d-b8dbdd1cf196 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.816362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639d253-6425-4870-bd0d-b8dbdd1cf196 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.817094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639d253-6425-4870-bd0d-b8dbdd1cf196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.820166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0639d253-6425-4870-bd0d-b8dbdd1cf196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:01.923533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704a224-2534-4c32-b78f-e81422cb0d6c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.940401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.942858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.943253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.943641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] PATH_INFO: `/v3/users/87a27812c5af4722ba37fd423824e9df` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.949966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a27812c5af4722ba37fd423824e9df)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.956653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.963001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.978658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639d253-6425-4870-bd0d-b8dbdd1cf196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.980083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639d253-6425-4870-bd0d-b8dbdd1cf196 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.004582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 863/1711] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:02.006093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05997032-a587-481f-85c2-168cf4507086 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.006388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05997032-a587-481f-85c2-168cf4507086 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.006667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05997032-a587-481f-85c2-168cf4507086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.062429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-05997032-a587-481f-85c2-168cf4507086 None None] MFA Rules not processed for user `256af8436dd842fdb4ebe90ec9be26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:02.063677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-05997032-a587-481f-85c2-168cf4507086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:02.094168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.115825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a27812c5af4722ba37fd423824e9df'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.150647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a27812c5af4722ba37fd423824e9df'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.181051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a27812c5af4722ba37fd423824e9df'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.183156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a27812c5af4722ba37fd423824e9df'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.202263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 864/1712] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:55:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:02.258606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a27812c5af4722ba37fd423824e9df'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.268986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05997032-a587-481f-85c2-168cf4507086 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.270767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-05997032-a587-481f-85c2-168cf4507086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:02.295470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 849/1713] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:01 2022] DELETE /identity/v3/users/87a27812c5af4722ba37fd423824e9df => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:02.296458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.301309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:02.309732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05997032-a587-481f-85c2-168cf4507086 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.465552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45a13a-ea9f-45ce-b301-8d2af8e18b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.466142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45a13a-ea9f-45ce-b301-8d2af8e18b3d service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.466431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45a13a-ea9f-45ce-b301-8d2af8e18b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.466714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45a13a-ea9f-45ce-b301-8d2af8e18b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.493776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45a13a-ea9f-45ce-b301-8d2af8e18b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.506235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45a13a-ea9f-45ce-b301-8d2af8e18b3d service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.518115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35c1ac-4746-4a6e-a03a-263381ff35c8 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.532700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.532700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.533125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.535519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] PATH_INFO: `/v3/projects/d422a40e978e4159823d860e0e1ac6a3` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.535868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 865/1714] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:02.552714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d422a40e978e4159823d860e0e1ac6a3)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.557489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.724055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd422a40e978e4159823d860e0e1ac6a3', 'request_id': 'req-de973fee-2b91-4914-b8fc-47dac6287ea1'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.732367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd422a40e978e4159823d860e0e1ac6a3', 'request_id': 'req-de973fee-2b91-4914-b8fc-47dac6287ea1'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.765183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd422a40e978e4159823d860e0e1ac6a3', 'request_id': 'req-de973fee-2b91-4914-b8fc-47dac6287ea1'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.791140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] The token cache is being invalidate because project d422a40e978e4159823d860e0e1ac6a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:02.791476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.792329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:02.794182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 850/1715] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:02 2022] DELETE /identity/v3/projects/d422a40e978e4159823d860e0e1ac6a3 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:02.799109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f3210-00d5-4c46-8d2b-408537a73832 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.799406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f3210-00d5-4c46-8d2b-408537a73832 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.799689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f3210-00d5-4c46-8d2b-408537a73832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.825356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d28f3210-00d5-4c46-8d2b-408537a73832 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:02.826662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d28f3210-00d5-4c46-8d2b-408537a73832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:02.859345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.861020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:02.880659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.000039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 866/1716] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:03.008117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56037f71-6d4b-4719-8e41-8e457e8e1a6f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.013653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56037f71-6d4b-4719-8e41-8e457e8e1a6f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.024494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56037f71-6d4b-4719-8e41-8e457e8e1a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.034839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de973fee-2b91-4914-b8fc-47dac6287ea1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.049232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.050770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.051481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.052108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] PATH_INFO: `/v3/users/9ea7096e5ac84095a0d6ff2b048afd91` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.059767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea7096e5ac84095a0d6ff2b048afd91)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.063967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.074281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-56037f71-6d4b-4719-8e41-8e457e8e1a6f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:03.075663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-56037f71-6d4b-4719-8e41-8e457e8e1a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:03.102716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 867/1717] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:03.106766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56037f71-6d4b-4719-8e41-8e457e8e1a6f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.126922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56037f71-6d4b-4719-8e41-8e457e8e1a6f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.143161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.144483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.145157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.145871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.148898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:03.149728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.155125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.193003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 868/1718] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:03.196166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.202874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:03.226066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea7096e5ac84095a0d6ff2b048afd91'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.235194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144a3c-1b49-4250-9ed9-93e4c291830c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.242139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea7096e5ac84095a0d6ff2b048afd91'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.250170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.251093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.251677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.252120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] PATH_INFO: `/v3/projects/fc2b850bf10549bfac5d1fe69b3e2b49` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.253756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2b850bf10549bfac5d1fe69b3e2b49)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.255887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.262811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea7096e5ac84095a0d6ff2b048afd91'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.263456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea7096e5ac84095a0d6ff2b048afd91'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.290140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea7096e5ac84095a0d6ff2b048afd91'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.300048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 851/1719] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:02 2022] DELETE /identity/v3/users/9ea7096e5ac84095a0d6ff2b048afd91 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:03.301331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.307982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:03.334356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.344440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2b850bf10549bfac5d1fe69b3e2b49', 'request_id': 'req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.360768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2b850bf10549bfac5d1fe69b3e2b49', 'request_id': 'req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.377315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2b850bf10549bfac5d1fe69b3e2b49', 'request_id': 'req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.413148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] The token cache is being invalidate because project fc2b850bf10549bfac5d1fe69b3e2b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:03.413439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.414590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8b2a6a91-c3e3-44a3-8002-adeb8fd77fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:03.416986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 869/1720] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:03 2022] DELETE /identity/v3/projects/fc2b850bf10549bfac5d1fe69b3e2b49 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:03.420866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109aa74-544d-4852-99aa-171ab38fea2d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.421460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109aa74-544d-4852-99aa-171ab38fea2d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.422763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109aa74-544d-4852-99aa-171ab38fea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.476367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6109aa74-544d-4852-99aa-171ab38fea2d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:03.479977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6109aa74-544d-4852-99aa-171ab38fea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:03.500196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d51e58-9cb2-428f-80df-857ab49acfcd admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.639560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.704505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 870/1721] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:03.708806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.709098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.709376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.714383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.716273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.716554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.716832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.717708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.719424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.770512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:03.780227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:03.809907 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 871/1722] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:03.810714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.817046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 852/1723] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:03 2022] POST /identity/v3/projects => generated 385 bytes in 516 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:03.818084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.819901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:03.827565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:04.019666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f7c08-c74a-4287-8079-8fc768d6beee admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.045011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea2835-366e-479f-ad85-48ebe64ea96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.045957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea2835-366e-479f-ad85-48ebe64ea96a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.046363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea2835-366e-479f-ad85-48ebe64ea96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.046737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea2835-366e-479f-ad85-48ebe64ea96a admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.047482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea2835-366e-479f-ad85-48ebe64ea96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.049361 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce90b9-76f0-4bf6-bd1d-f076956df2ba None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.053339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea2835-366e-479f-ad85-48ebe64ea96a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.103857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 853/1724] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:03 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:04.103857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29002087-9d9f-41a9-a711-ee0ec736e30f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.103857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29002087-9d9f-41a9-a711-ee0ec736e30f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.103857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29002087-9d9f-41a9-a711-ee0ec736e30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.151316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-29002087-9d9f-41a9-a711-ee0ec736e30f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:04.152711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-29002087-9d9f-41a9-a711-ee0ec736e30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:04.162008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.162888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.163177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.163494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.178641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 854/1725] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:04.179852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29002087-9d9f-41a9-a711-ee0ec736e30f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.191676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.193174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.205359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29002087-9d9f-41a9-a711-ee0ec736e30f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.222412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 872/1726] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:04.223287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.232507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.239293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.239841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.240301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.241180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.244396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.251618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a711943-1f01-4de6-b12b-bf0d08fa3386 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.268683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.269322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.269603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.269878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.271200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:04.271473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.273158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.291707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 855/1727] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:04.296929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.314298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 873/1728] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:04.315883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.330830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d791ad9-dc4b-409c-920e-b1a2518f32c9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.348837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cb987-12a3-4feb-a8b5-c1f48befed1a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.349821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.350616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.351323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.353761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.354702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.362959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.368526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29532f7-85df-4f38-8dd1-16c7228883f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.375943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29532f7-85df-4f38-8dd1-16c7228883f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.376286 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29532f7-85df-4f38-8dd1-16c7228883f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.376579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29532f7-85df-4f38-8dd1-16c7228883f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.379748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29532f7-85df-4f38-8dd1-16c7228883f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.384517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29532f7-85df-4f38-8dd1-16c7228883f9 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.413930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.456054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 856/1729] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:04.460890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.466469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 874/1730] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:04 2022] POST /identity/v3/projects => generated 407 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:04.470918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e0bd8-fac9-458a-825e-7a0274af7e77 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.471228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e0bd8-fac9-458a-825e-7a0274af7e77 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.471488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e0bd8-fac9-458a-825e-7a0274af7e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.488306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7714a-a828-4d47-9695-0ba37bb7ef14 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.657569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.658930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.659323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.659711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] PATH_INFO: `/v3/projects/955b815c57e54eeb86273eedbd7fcacb/users/4271d9de61bc4031b69b38d298f3e8b2/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.681281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-376e0bd8-fac9-458a-825e-7a0274af7e77 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:04.682548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-376e0bd8-fac9-458a-825e-7a0274af7e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:04.698531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955b815c57e54eeb86273eedbd7fcacb, user_id=4271d9de61bc4031b69b38d298f3e8b2, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.701972 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.733973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 857/1731] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:04 2022] PUT /identity/v3/projects/955b815c57e54eeb86273eedbd7fcacb/users/4271d9de61bc4031b69b38d298f3e8b2/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:04.733973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.751781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.933121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 875/1732] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:55:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:04.934751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae380ab-7ebd-47c8-b7e4-f2d4a28b1092 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.938484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae380ab-7ebd-47c8-b7e4-f2d4a28b1092 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.939747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae380ab-7ebd-47c8-b7e4-f2d4a28b1092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.007521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-dae380ab-7ebd-47c8-b7e4-f2d4a28b1092 None None] MFA Rules not processed for user `7a701149974b45db9f782bc5a232aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:05.008992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-dae380ab-7ebd-47c8-b7e4-f2d4a28b1092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.015313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904946d-f231-46b0-bda8-c8c799f78487 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.030456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f82a8d-5e40-4f53-8916-29562279a63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.031488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f82a8d-5e40-4f53-8916-29562279a63b admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.031812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f82a8d-5e40-4f53-8916-29562279a63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.032084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f82a8d-5e40-4f53-8916-29562279a63b admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.032743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f82a8d-5e40-4f53-8916-29562279a63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.038167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f82a8d-5e40-4f53-8916-29562279a63b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.092429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 858/1733] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:04 2022] POST /identity/v3/users => generated 304 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:05.094646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.095047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.095583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.145735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 876/1734] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:05.150226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258f8d1-dcc0-4c6c-b038-a668cbe85097 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.150602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258f8d1-dcc0-4c6c-b038-a668cbe85097 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.150793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258f8d1-dcc0-4c6c-b038-a668cbe85097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.154067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] MFA Rules not processed for user `c7fac765624c4e0b85b01f356a14a1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:05.158144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.229084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3258f8d1-dcc0-4c6c-b038-a668cbe85097 None None] MFA Rules not processed for user `4271d9de61bc4031b69b38d298f3e8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:05.230376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3258f8d1-dcc0-4c6c-b038-a668cbe85097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.368808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.370204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 859/1735] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:05.373440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.376087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.394255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 877/1736] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:05.398779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258f8d1-dcc0-4c6c-b038-a668cbe85097 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.430817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be46d9c-0b12-43ea-957b-29c7505614a6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.437066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258f8d1-dcc0-4c6c-b038-a668cbe85097 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.440050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.441553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.441834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.442116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] PATH_INFO: `/v3/projects/6840c8d2676542ecbed0e9a237e489f5` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.445390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6840c8d2676542ecbed0e9a237e489f5)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.447027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.447552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.447835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.448112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.448793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.453714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.455937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.477401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 878/1737] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:05.477401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.477886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.526055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6840c8d2676542ecbed0e9a237e489f5', 'request_id': 'req-47e26463-54f6-4fdd-803a-90ae4ba43403'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:05.542344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6840c8d2676542ecbed0e9a237e489f5', 'request_id': 'req-47e26463-54f6-4fdd-803a-90ae4ba43403'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:05.558489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6840c8d2676542ecbed0e9a237e489f5', 'request_id': 'req-47e26463-54f6-4fdd-803a-90ae4ba43403'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:05.593120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] The token cache is being invalidate because project 6840c8d2676542ecbed0e9a237e489f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:05.593626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:05.594498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:05.596658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 860/1738] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:05 2022] DELETE /identity/v3/projects/6840c8d2676542ecbed0e9a237e489f5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:05.597581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.600962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.653421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.720977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5aac2-a016-4ce8-84c1-b540386046e4 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.835951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e26463-54f6-4fdd-803a-90ae4ba43403 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.860854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.862381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.863791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.866627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.869845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.876014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.895000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145f6d-5523-48dc-b785-8c5b78ee9d5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.896277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145f6d-5523-48dc-b785-8c5b78ee9d5c service swift] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.896627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145f6d-5523-48dc-b785-8c5b78ee9d5c service swift] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.896906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145f6d-5523-48dc-b785-8c5b78ee9d5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.899034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b0145f6d-5523-48dc-b785-8c5b78ee9d5c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.960089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 861/1739] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:05 2022] POST /identity/v3/projects => generated 387 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:05.960288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.966482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.996885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145f6d-5523-48dc-b785-8c5b78ee9d5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.999466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145f6d-5523-48dc-b785-8c5b78ee9d5c service swift] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.012422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 879/1740] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:55:06.014394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fcb94-451f-4c49-9645-59666ae84010 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.015387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fcb94-451f-4c49-9645-59666ae84010 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.015387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fcb94-451f-4c49-9645-59666ae84010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.044114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-932fcb94-451f-4c49-9645-59666ae84010 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:06.045764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-932fcb94-451f-4c49-9645-59666ae84010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:06.051869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:06.180006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 880/1741] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:06.182640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fcb94-451f-4c49-9645-59666ae84010 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.194934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902574bf-77ac-4033-8b64-ce3494338a4c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.211378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fcb94-451f-4c49-9645-59666ae84010 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.260909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414d24c-dfb6-4d39-a425-a9af2f75d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.261227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.262301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d24c-dfb6-4d39-a425-a9af2f75d0cc admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.262419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.262592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d24c-dfb6-4d39-a425-a9af2f75d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.262683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.262864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d24c-dfb6-4d39-a425-a9af2f75d0cc admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.262953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.263537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414d24c-dfb6-4d39-a425-a9af2f75d0cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.263634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.273497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414d24c-dfb6-4d39-a425-a9af2f75d0cc admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.320005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.369360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 881/1742] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:06 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:06.369360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec879c-8a82-4f8c-aa74-7beef4d3ca98 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.377990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec879c-8a82-4f8c-aa74-7beef4d3ca98 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.378421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec879c-8a82-4f8c-aa74-7beef4d3ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.379741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 862/1743] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:06.387564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.428628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143a2d7-e7f4-49ae-9e66-fcf0fe7420e7 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.442973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.443614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.443895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.444189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.444849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.447604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-daec879c-8a82-4f8c-aa74-7beef4d3ca98 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:06.452807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-daec879c-8a82-4f8c-aa74-7beef4d3ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:06.453961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.479965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 882/1744] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:06.481198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec879c-8a82-4f8c-aa74-7beef4d3ca98 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.488672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 863/1745] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:06.488873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.539422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec879c-8a82-4f8c-aa74-7beef4d3ca98 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.557385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596baa0-8c3c-49cd-8833-9bd4d1e6835f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.561949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.565835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.568629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.571453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] PATH_INFO: `/v3/projects/caa481bfbf834e078254e3d45bfb6ba1/users/b7c46d0590ee4389ac76c661baf4dbcf/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.585081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.590015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.590296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.590574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.593609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:06.593919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.612589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.655147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa481bfbf834e078254e3d45bfb6ba1, user_id=b7c46d0590ee4389ac76c661baf4dbcf, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.670332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.678176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 864/1746] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:06.678578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.725242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 883/1747] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:06 2022] PUT /identity/v3/projects/caa481bfbf834e078254e3d45bfb6ba1/users/b7c46d0590ee4389ac76c661baf4dbcf/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:06.728469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.748945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d986f-56ef-445e-921a-5ac32f70f840 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.925974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22254bb-4281-4f29-a3ea-1e53420c49ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.927817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22254bb-4281-4f29-a3ea-1e53420c49ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.928102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22254bb-4281-4f29-a3ea-1e53420c49ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.928414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e7477-18f6-4971-8d2d-611a3f104d84 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.929636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22254bb-4281-4f29-a3ea-1e53420c49ce admin admin] PATH_INFO: `/v3/projects/b60f3835792741c78e844b9380d2f819/users/35a25b17d40a4773897376691dc4389c/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.939885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.942257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.942546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.942935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.946483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.951759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.974207 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22254bb-4281-4f29-a3ea-1e53420c49ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60f3835792741c78e844b9380d2f819, user_id=35a25b17d40a4773897376691dc4389c, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.977294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22254bb-4281-4f29-a3ea-1e53420c49ce admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.032771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 865/1748] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:06 2022] PUT /identity/v3/projects/b60f3835792741c78e844b9380d2f819/users/35a25b17d40a4773897376691dc4389c/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:07.036512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec480-1e5e-4304-9100-e532fa26e900 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.036832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec480-1e5e-4304-9100-e532fa26e900 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.037111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec480-1e5e-4304-9100-e532fa26e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.051177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 884/1749] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:06 2022] POST /identity/v3/projects => generated 371 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:07.052776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.061008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.145276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1a3ec480-1e5e-4304-9100-e532fa26e900 None None] MFA Rules not processed for user `b7c46d0590ee4389ac76c661baf4dbcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:07.152739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1a3ec480-1e5e-4304-9100-e532fa26e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:07.299601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 866/1750] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:07 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:07.301014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec480-1e5e-4304-9100-e532fa26e900 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.329274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.391540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9fd1f-f194-4210-8648-ca121f19fc8e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.455058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.459219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.460813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.461325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.466737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.480270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.543814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 885/1751] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:07.545508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.577435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e052-251c-46d4-be65-7174c320d993 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.599226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.600939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.602974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.604855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.606715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.613246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.653687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec480-1e5e-4304-9100-e532fa26e900 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.672452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.673124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.673498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.675339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.676320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.683265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.713265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 886/1752] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:07 2022] POST /identity/v3/projects => generated 407 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:07.714338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.760661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 867/1753] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:07 2022] POST /identity/v3/users => generated 286 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:07.765363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.790212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249a024-0953-4289-afb9-7acdfd029768 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.954344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb96af-758a-4866-9562-b1c1129ea52a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.971101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.974992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.975284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.975572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] PATH_INFO: `/v3/projects/b60f3835792741c78e844b9380d2f819/users/35a25b17d40a4773897376691dc4389c/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.986534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60f3835792741c78e844b9380d2f819, user_id=35a25b17d40a4773897376691dc4389c, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.989107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.025676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8d948-4f6f-44c3-805c-f98bb10edcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.030748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8d948-4f6f-44c3-805c-f98bb10edcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.034561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8d948-4f6f-44c3-805c-f98bb10edcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.037373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8d948-4f6f-44c3-805c-f98bb10edcd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.037859 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 887/1754] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:07 2022] PUT /identity/v3/projects/b60f3835792741c78e844b9380d2f819/users/35a25b17d40a4773897376691dc4389c/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:08.038720 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.039389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8d948-4f6f-44c3-805c-f98bb10edcd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.050427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8d948-4f6f-44c3-805c-f98bb10edcd4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.150782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 868/1755] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:07 2022] POST /identity/v3/users => generated 310 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:08.155717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457008-382f-43ff-8fff-aee76a75af2b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.161401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457008-382f-43ff-8fff-aee76a75af2b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.161840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457008-382f-43ff-8fff-aee76a75af2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.252876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.297512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-14457008-382f-43ff-8fff-aee76a75af2b None None] MFA Rules not processed for user `35a25b17d40a4773897376691dc4389c`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:08.323960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-14457008-382f-43ff-8fff-aee76a75af2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:08.361536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fef328-c749-4f6b-b430-751c3d4202b1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.378616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.379511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.379947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.380597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.381568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.383349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.423211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 888/1756] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:08.424903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.460228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044069a5-5b1c-4273-84c4-d38a5c4d2aaa admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.490105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.490910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.491346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.491763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.492595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.494357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.515630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 869/1757] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:08.516744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14457008-382f-43ff-8fff-aee76a75af2b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.530999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14457008-382f-43ff-8fff-aee76a75af2b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.536472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 889/1758] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:08.541026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.542354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.544282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.544703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.545311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.546188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:08.548406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.550132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.567111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 870/1759] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:08.568224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.580039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b5e0e-6f74-485f-b947-6af2a2f2369d admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.585663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.586618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.587822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.588422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] PATH_INFO: `/v3/projects/1b933318ecbe42818c3e91eadd15cd8d/users/509faa1cbc874eeb950ae1067c922ca6/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.625998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b933318ecbe42818c3e91eadd15cd8d, user_id=509faa1cbc874eeb950ae1067c922ca6, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.635199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.699711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 871/1760] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:08 2022] PUT /identity/v3/projects/1b933318ecbe42818c3e91eadd15cd8d/users/509faa1cbc874eeb950ae1067c922ca6/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:08.699711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.733201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.787631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb254b8-63b1-4e82-a347-fb8646481650 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.822809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921a27-7d06-4274-9592-130337345473 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.829959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.830791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.831262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.831682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] PATH_INFO: `/v3/projects/02d19faf494248a08030749766b5d633/users/612336b0bfa64842ab15c0a9ba985dd4/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.856035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d19faf494248a08030749766b5d633, user_id=612336b0bfa64842ab15c0a9ba985dd4, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.859662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.899921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 872/1761] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:08 2022] PUT /identity/v3/projects/02d19faf494248a08030749766b5d633/users/612336b0bfa64842ab15c0a9ba985dd4/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:08.915610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.970343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.977993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.978640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.979135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.074923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.151437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344c840-3738-4ece-bd4c-13cf14640676 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.170212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3e02c-5c64-4695-9856-ee71a78854ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.171642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3e02c-5c64-4695-9856-ee71a78854ae admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.171940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3e02c-5c64-4695-9856-ee71a78854ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.172249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3e02c-5c64-4695-9856-ee71a78854ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.172922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3e02c-5c64-4695-9856-ee71a78854ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.178552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3e02c-5c64-4695-9856-ee71a78854ae admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.198775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.200300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.229321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 873/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:09.233834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 890/1763] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:09.238699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.264939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110e79a-e76c-4b52-ad4f-7455eca663e1 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.265626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110e79a-e76c-4b52-ad4f-7455eca663e1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.266808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110e79a-e76c-4b52-ad4f-7455eca663e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.283763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcaa21-54d1-40a8-9a7b-b66f651fdefb service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.302213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.302913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.303196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.303757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] PATH_INFO: `/v3/projects/1b933318ecbe42818c3e91eadd15cd8d/users/509faa1cbc874eeb950ae1067c922ca6/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.314344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b933318ecbe42818c3e91eadd15cd8d, user_id=509faa1cbc874eeb950ae1067c922ca6, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.317075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.402741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 891/1764] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:09 2022] PUT /identity/v3/projects/1b933318ecbe42818c3e91eadd15cd8d/users/509faa1cbc874eeb950ae1067c922ca6/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:09.411283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.431098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2110e79a-e76c-4b52-ad4f-7455eca663e1 None None] MFA Rules not processed for user `612336b0bfa64842ab15c0a9ba985dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:09.432370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2110e79a-e76c-4b52-ad4f-7455eca663e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:09.702283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5ff1b-cd37-4db5-ae89-825172263000 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.721795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.727023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.727834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.732689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] PATH_INFO: `/v3/users/4271d9de61bc4031b69b38d298f3e8b2` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.736969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4271d9de61bc4031b69b38d298f3e8b2)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.747248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.781475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 874/1765] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:09.782904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.783341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.783621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.887640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.935518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] MFA Rules not processed for user `256af8436dd842fdb4ebe90ec9be26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:09.937076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4271d9de61bc4031b69b38d298f3e8b2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:09.939056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4271d9de61bc4031b69b38d298f3e8b2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:09.946022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:09.962894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4271d9de61bc4031b69b38d298f3e8b2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:09.966972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4271d9de61bc4031b69b38d298f3e8b2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:10.000526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e4997d9c-288c-4492-bfc4-b0576fd0ac9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4271d9de61bc4031b69b38d298f3e8b2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:10.021134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 892/1766] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:09 2022] DELETE /identity/v3/users/4271d9de61bc4031b69b38d298f3e8b2 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:10.022915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd7b5-ebda-414e-a240-46f72401f80d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.022915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd7b5-ebda-414e-a240-46f72401f80d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.023857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd7b5-ebda-414e-a240-46f72401f80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.143048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b80fd7b5-ebda-414e-a240-46f72401f80d None None] MFA Rules not processed for user `509faa1cbc874eeb950ae1067c922ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:10.158232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b80fd7b5-ebda-414e-a240-46f72401f80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:10.167235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 875/1767] 10.222.0.48 () {58 vars in 1015 bytes} [Wed Aug 10 16:55:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:10.169775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.173830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:10.209260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79e2d8-e7d4-4bf4-91b1-616d8f4a8d89 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.382084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 893/1768] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:10 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:10.382084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.382084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.382084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.408039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:10.410021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:10.504241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 894/1769] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:10.506122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.508102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:10.525117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.527316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.528682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.529361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.532197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.537594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.566977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.627278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08956915-1682-42b1-81c3-afde5828dde5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.654260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.656922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.657668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.657954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.679719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 876/1770] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:10 2022] POST /identity/v3/projects => generated 373 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:10.679719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.689773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:10.702451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.716224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.722397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.800729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 895/1771] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:10.806630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.833177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92459d76-7563-4fc2-91c4-f9bbcaea633a service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.877903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badb8f3-c15b-400e-a64d-6be3d28ce985 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.886466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.888465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.888907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.889363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.082904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.089711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.149112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 877/1772] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:11.153609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.232902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.234274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.236023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.236742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.523470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.537672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.566790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ef03b-e3ab-4697-86a8-82ad60c01302 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.583432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 896/1773] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:11.586710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.590371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.590682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.591019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.592590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.598620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.677241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 878/1774] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:11 2022] POST /identity/v3/users => generated 293 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:11.681794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.713544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7e80c-025c-4c32-978a-46a4f16dd54a service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.737410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.738047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.738329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.738601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.742555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.746601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.755700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.774780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36fbfa-647b-4081-bff4-d7df50ab0048 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.791899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.793423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.793709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.793982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.800413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:11.802664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 897/1775] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:11.804092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.842539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d942b3-0bd3-4c1d-8d3c-48bbea07ff26 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.858387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.859102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.859379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.859625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] PATH_INFO: `/v3/projects/672652cb49c1427ab2f9396439b238a5/users/ccc17e1851364e3ca24e593b2c0ff46e/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.902358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672652cb49c1427ab2f9396439b238a5, user_id=ccc17e1851364e3ca24e593b2c0ff46e, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.905092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.940414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:11.955320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 898/1776] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:11 2022] PUT /identity/v3/projects/672652cb49c1427ab2f9396439b238a5/users/ccc17e1851364e3ca24e593b2c0ff46e/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:11.961485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.001804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.003316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.042404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 879/1777] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:55:12.127662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:12.134922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496ef56-ff0d-407a-82b7-2bf55a268a36 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.149960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8079c1-bc0b-4758-9bd8-cbb47a8f9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.150604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8079c1-bc0b-4758-9bd8-cbb47a8f9eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.150870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8079c1-bc0b-4758-9bd8-cbb47a8f9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.151425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8079c1-bc0b-4758-9bd8-cbb47a8f9eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.153658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8079c1-bc0b-4758-9bd8-cbb47a8f9eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.155317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8079c1-bc0b-4758-9bd8-cbb47a8f9eb8 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.178921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 899/1778] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:12.182779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.200458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a6583-7ebc-4907-b380-f23aafa033f7 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.216138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.217363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.217619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.217880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] PATH_INFO: `/v3/projects/672652cb49c1427ab2f9396439b238a5/users/ccc17e1851364e3ca24e593b2c0ff46e/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.224415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=672652cb49c1427ab2f9396439b238a5, user_id=ccc17e1851364e3ca24e593b2c0ff46e, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.230662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.261848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 880/1779] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:12 2022] PUT /identity/v3/projects/672652cb49c1427ab2f9396439b238a5/users/ccc17e1851364e3ca24e593b2c0ff46e/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:12.272363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d287a38-92dc-4fda-9b66-5c4b0eb2f2dc None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.272732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d287a38-92dc-4fda-9b66-5c4b0eb2f2dc None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.273050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d287a38-92dc-4fda-9b66-5c4b0eb2f2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.333572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-7d287a38-92dc-4fda-9b66-5c4b0eb2f2dc None None] MFA Rules not processed for user `ccc17e1851364e3ca24e593b2c0ff46e`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:12.339283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7d287a38-92dc-4fda-9b66-5c4b0eb2f2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:12.510058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 900/1780] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:12.534846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.538241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:12.791366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dda4d-0947-4f71-be22-9c745b1c1a9c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.813711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed5b12-dd1e-46db-91a6-fd8953361198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.814268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed5b12-dd1e-46db-91a6-fd8953361198 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.814574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed5b12-dd1e-46db-91a6-fd8953361198 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.814811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed5b12-dd1e-46db-91a6-fd8953361198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.855770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed5b12-dd1e-46db-91a6-fd8953361198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.855770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed5b12-dd1e-46db-91a6-fd8953361198 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.885233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 881/1781] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:13.170662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d287a38-92dc-4fda-9b66-5c4b0eb2f2dc None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.185012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d287a38-92dc-4fda-9b66-5c4b0eb2f2dc None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.212918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884b187-7fd6-4bd2-80f9-04fa258f1dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.213876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884b187-7fd6-4bd2-80f9-04fa258f1dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.214156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884b187-7fd6-4bd2-80f9-04fa258f1dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.214514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884b187-7fd6-4bd2-80f9-04fa258f1dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.222928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e884b187-7fd6-4bd2-80f9-04fa258f1dd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:13.406219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884b187-7fd6-4bd2-80f9-04fa258f1dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.408287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884b187-7fd6-4bd2-80f9-04fa258f1dd4 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.453588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 901/1782] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:13.630415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee3e3e-cb54-4bde-807b-adbf8e02d8ca None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.630928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee3e3e-cb54-4bde-807b-adbf8e02d8ca None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.631337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee3e3e-cb54-4bde-807b-adbf8e02d8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.672819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ceee3e3e-cb54-4bde-807b-adbf8e02d8ca None None] MFA Rules not processed for user `96495ad2ad764ba3a0fb1e3bc28fd8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:13.674430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ceee3e3e-cb54-4bde-807b-adbf8e02d8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:13.875797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 882/1783] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:15.072433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778ad57-f14e-4fdc-b9b6-2dd4afdaa795 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.072937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778ad57-f14e-4fdc-b9b6-2dd4afdaa795 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.073011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778ad57-f14e-4fdc-b9b6-2dd4afdaa795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.173465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f778ad57-f14e-4fdc-b9b6-2dd4afdaa795 None None] MFA Rules not processed for user `77b8847d10744ec98e0ed7a3f1e991ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:15.175153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f778ad57-f14e-4fdc-b9b6-2dd4afdaa795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:15.469389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 902/1784] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:15.659180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442954e-306b-43ce-9239-23168b3695d1 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.659761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442954e-306b-43ce-9239-23168b3695d1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.659903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442954e-306b-43ce-9239-23168b3695d1 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.662362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 883/1785] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 16:55:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:15.679626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442954e-306b-43ce-9239-23168b3695d1 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.680973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.681357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.681522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.688691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c442954e-306b-43ce-9239-23168b3695d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:15.726598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.781013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:15.782501 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:15.889426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442954e-306b-43ce-9239-23168b3695d1 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.895943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.896848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.897375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.898192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] PATH_INFO: `/v3/projects/955b815c57e54eeb86273eedbd7fcacb` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.900502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=955b815c57e54eeb86273eedbd7fcacb)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.903993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.954836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.000129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 903/1786] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 16:55:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:16.000129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.000129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:16.015827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955b815c57e54eeb86273eedbd7fcacb', 'request_id': 'req-ea6a2c3d-282e-4359-9f25-0c396774f161'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.035542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955b815c57e54eeb86273eedbd7fcacb', 'request_id': 'req-ea6a2c3d-282e-4359-9f25-0c396774f161'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.046978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955b815c57e54eeb86273eedbd7fcacb', 'request_id': 'req-ea6a2c3d-282e-4359-9f25-0c396774f161'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.055444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7271e-c1eb-4da5-9107-33d59ec5961f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.086421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] The token cache is being invalidate because project 955b815c57e54eeb86273eedbd7fcacb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:16.086611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.089318 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.091042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 884/1787] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:15 2022] DELETE /identity/v3/projects/955b815c57e54eeb86273eedbd7fcacb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:16.098836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.100654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:16.244472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.269653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.270027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.270386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.274657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:16.274887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a2c3d-282e-4359-9f25-0c396774f161 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.297916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.298238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.298509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.298700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] PATH_INFO: `/v3/users/35a25b17d40a4773897376691dc4389c` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.300024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a25b17d40a4773897376691dc4389c)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.302302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.430068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a25b17d40a4773897376691dc4389c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.460522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.462604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.468516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a25b17d40a4773897376691dc4389c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.477081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 904/1788] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:16.480923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a25b17d40a4773897376691dc4389c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.481770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a25b17d40a4773897376691dc4389c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.504703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a25b17d40a4773897376691dc4389c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:16.512918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 885/1789] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:16 2022] DELETE /identity/v3/users/35a25b17d40a4773897376691dc4389c => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:16.616495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.618881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:16.648506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1816ddc-60d1-4e9a-935c-09f95f475990 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.727588 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.788989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b03c0-6dcd-4184-b30d-455ec21783c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.790024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b03c0-6dcd-4184-b30d-455ec21783c8 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.790524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b03c0-6dcd-4184-b30d-455ec21783c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.791011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b03c0-6dcd-4184-b30d-455ec21783c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.793686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-eb1b03c0-6dcd-4184-b30d-455ec21783c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:16.812851 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b03c0-6dcd-4184-b30d-455ec21783c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.815903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b03c0-6dcd-4184-b30d-455ec21783c8 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.832987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 905/1790] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:17.282137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.283880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:17.372205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.454068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55c36b-359c-4ce1-8fd8-5c8beaf555f7 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.454986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55c36b-359c-4ce1-8fd8-5c8beaf555f7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.455434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55c36b-359c-4ce1-8fd8-5c8beaf555f7 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.457049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 906/1791] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 16:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:17.463088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.463531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.463959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.464457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733a51-4fbc-4ce2-aabc-7879a17b5944 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.470870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.471647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.472037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.472482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] PATH_INFO: `/v3/projects/b60f3835792741c78e844b9380d2f819` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.474919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b60f3835792741c78e844b9380d2f819)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.477152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.493749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:17.496802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:17.532045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 907/1792] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 16:55:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:55:17.533126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.534961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:17.550546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f8a66-63c8-4f5c-946d-8ea74cdd11d5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.552099 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b60f3835792741c78e844b9380d2f819', 'request_id': 'req-d8bc338b-d630-42bf-9441-152bc866068f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:17.565147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b60f3835792741c78e844b9380d2f819', 'request_id': 'req-d8bc338b-d630-42bf-9441-152bc866068f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:17.574952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b60f3835792741c78e844b9380d2f819', 'request_id': 'req-d8bc338b-d630-42bf-9441-152bc866068f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:17.597006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] The token cache is being invalidate because project b60f3835792741c78e844b9380d2f819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:17.597588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:17.599084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d8bc338b-d630-42bf-9441-152bc866068f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:17.601426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 886/1793] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:17 2022] DELETE /identity/v3/projects/b60f3835792741c78e844b9380d2f819 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:17.606819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd216e0-e745-4e03-9a33-eac9631f477b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.607810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd216e0-e745-4e03-9a33-eac9631f477b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.608952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd216e0-e745-4e03-9a33-eac9631f477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.643165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6cd216e0-e745-4e03-9a33-eac9631f477b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:17.654247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6cd216e0-e745-4e03-9a33-eac9631f477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:17.659336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9efb-7225-459f-9b66-140e83990207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.659898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9efb-7225-459f-9b66-140e83990207 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.662414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9efb-7225-459f-9b66-140e83990207 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.662872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9efb-7225-459f-9b66-140e83990207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.668490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0f9e9efb-7225-459f-9b66-140e83990207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:17.685882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e9efb-7225-459f-9b66-140e83990207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.687663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e9efb-7225-459f-9b66-140e83990207 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.702624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 908/1794] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:17.785857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 887/1795] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:17.787689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33341-a295-4a8e-884f-4fc6a9beaf93 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.788222 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33341-a295-4a8e-884f-4fc6a9beaf93 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.788792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33341-a295-4a8e-884f-4fc6a9beaf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.819806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-abd33341-a295-4a8e-884f-4fc6a9beaf93 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:17.821169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-abd33341-a295-4a8e-884f-4fc6a9beaf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:17.834779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 909/1796] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:17.835830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd216e0-e745-4e03-9a33-eac9631f477b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.847745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd216e0-e745-4e03-9a33-eac9631f477b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.852917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.853695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.854344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.854344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.855016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:17.855238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.856913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.878724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 888/1797] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:17.879834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd33341-a295-4a8e-884f-4fc6a9beaf93 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.889821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd33341-a295-4a8e-884f-4fc6a9beaf93 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.895575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.896224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.896536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.896816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.897459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.899358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.932325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 910/1798] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:17 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:17.936643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.952657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa35e-3742-424b-9b69-a6a1f1f9a122 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.006093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.030355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.032118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.032692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.033393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.035142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.039549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.077662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 889/1799] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:17 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:18.082580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.093714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b282c5-baee-40d6-9ee9-d0ccd121e783 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.098682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.099309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.099587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.099864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.100586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.102572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.117704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 911/1800] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:18.118663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.131207 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2401f-e4a6-441a-823b-380625b52f82 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.144471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b559c32-6ab3-46c7-ad5c-48da076d2b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.145282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b559c32-6ab3-46c7-ad5c-48da076d2b81 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.145763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b559c32-6ab3-46c7-ad5c-48da076d2b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.145897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b559c32-6ab3-46c7-ad5c-48da076d2b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.146616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b559c32-6ab3-46c7-ad5c-48da076d2b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.148354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b559c32-6ab3-46c7-ad5c-48da076d2b81 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.162935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 890/1801] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:55:18.166785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.195261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb835c-1a4f-408a-97f7-28004aa9ad6e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.212645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.213297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.213577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.213852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] PATH_INFO: `/v3/projects/16c708ab63b243fea7d212f3a6a2d875/users/9f800aac23ce487a8c95a0f64898059a/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.236733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c708ab63b243fea7d212f3a6a2d875, user_id=9f800aac23ce487a8c95a0f64898059a, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.240677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.278208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 912/1802] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:18 2022] PUT /identity/v3/projects/16c708ab63b243fea7d212f3a6a2d875/users/9f800aac23ce487a8c95a0f64898059a/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:18.279003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743bebb-1c8f-41d1-93f8-b484768e6649 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.279156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743bebb-1c8f-41d1-93f8-b484768e6649 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.279416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743bebb-1c8f-41d1-93f8-b484768e6649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.335287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-0743bebb-1c8f-41d1-93f8-b484768e6649 None None] MFA Rules not processed for user `9f800aac23ce487a8c95a0f64898059a`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:18.336601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0743bebb-1c8f-41d1-93f8-b484768e6649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:18.433006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 891/1803] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:18.434731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.487058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.530530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fe7e2-77bf-494b-bc9e-4510f9b14359 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.540470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce991f-8b61-4f3b-a67e-9d941fdf0434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.541914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce991f-8b61-4f3b-a67e-9d941fdf0434 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.542236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce991f-8b61-4f3b-a67e-9d941fdf0434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.543857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce991f-8b61-4f3b-a67e-9d941fdf0434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.546074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce991f-8b61-4f3b-a67e-9d941fdf0434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.549414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce991f-8b61-4f3b-a67e-9d941fdf0434 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.585133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 913/1804] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:18 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:18.587733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743bebb-1c8f-41d1-93f8-b484768e6649 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.602873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743bebb-1c8f-41d1-93f8-b484768e6649 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.683473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa88ce8-fd8e-4551-8f6c-97625259ad86 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.683765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa88ce8-fd8e-4551-8f6c-97625259ad86 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.684044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa88ce8-fd8e-4551-8f6c-97625259ad86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.687758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.688961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.689793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.690394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.691179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.693312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.713506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-aaa88ce8-fd8e-4551-8f6c-97625259ad86 None None] MFA Rules not processed for user `77b8847d10744ec98e0ed7a3f1e991ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:18.715591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aaa88ce8-fd8e-4551-8f6c-97625259ad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:18.728215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 892/1805] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:18 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:18.730047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.744683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc466e-b92a-4b98-9fdc-d3a1e9650eff admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.751259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.752586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.752717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.752982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.753666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.755738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.768548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 893/1806] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:55:18.775489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.790237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fd8bb-14c6-46b6-959b-71737f7ddd69 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.796183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 914/1807] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:18.798713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486e20f-ade5-47e9-a67d-3e5e2bf12892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.799345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e20f-ade5-47e9-a67d-3e5e2bf12892 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.799624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e20f-ade5-47e9-a67d-3e5e2bf12892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.799897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e20f-ade5-47e9-a67d-3e5e2bf12892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.800687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486e20f-ade5-47e9-a67d-3e5e2bf12892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.802436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486e20f-ade5-47e9-a67d-3e5e2bf12892 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.814502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 894/1808] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:55:18.818675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa88ce8-fd8e-4551-8f6c-97625259ad86 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.832047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa88ce8-fd8e-4551-8f6c-97625259ad86 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.838413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.839131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.839999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.840257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] PATH_INFO: `/v3/projects/48a00db9285e478fbea16647ecb77663/users/500ad182424b477181da6a11945301e7/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.858327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a00db9285e478fbea16647ecb77663, user_id=500ad182424b477181da6a11945301e7, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.862080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.888305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 915/1809] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:18 2022] PUT /identity/v3/projects/48a00db9285e478fbea16647ecb77663/users/500ad182424b477181da6a11945301e7/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:55:18.898948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e31f-a2da-42e3-8806-1f09b5ddd2c1 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.899303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e31f-a2da-42e3-8806-1f09b5ddd2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.899676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e31f-a2da-42e3-8806-1f09b5ddd2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.939663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5f44e31f-a2da-42e3-8806-1f09b5ddd2c1 None None] MFA Rules not processed for user `500ad182424b477181da6a11945301e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:18.940524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5f44e31f-a2da-42e3-8806-1f09b5ddd2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:18.952534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.052648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 895/1810] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:19.053982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.163087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23596b8-cddb-418c-a788-aaa204413ebd admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.175997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.176661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.176961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.177228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.177868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.180125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.259376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 916/1811] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:19 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:55:19.263416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e31f-a2da-42e3-8806-1f09b5ddd2c1 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.282625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e31f-a2da-42e3-8806-1f09b5ddd2c1 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.454593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.455411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.455520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.455767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.456419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.458632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.487838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 896/1812] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:19 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:19.491634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.504156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065312f-c8d4-4fb0-83f9-4522f65409ef admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.509562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.510120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.510403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.510673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.512019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.513844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.528583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 917/1813] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:19.532769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.546273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9f29c-d8b4-46be-b6cf-c729244febdc admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.551685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.552519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.552799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.553084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] PATH_INFO: `/v3/projects/7f52ecaf83e441dbb4fc6ff1a0f935cf/users/ed3568023ead4f41bfe628c070ebf00f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.572521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f52ecaf83e441dbb4fc6ff1a0f935cf, user_id=ed3568023ead4f41bfe628c070ebf00f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.576389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.598765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 897/1814] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:19 2022] PUT /identity/v3/projects/7f52ecaf83e441dbb4fc6ff1a0f935cf/users/ed3568023ead4f41bfe628c070ebf00f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:19.603343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.627183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196889-5a46-46a9-a563-a52910a0011f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.655059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.833214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6eb5b-a50c-411c-9b68-7089fa821e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.834000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6eb5b-a50c-411c-9b68-7089fa821e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.834384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6eb5b-a50c-411c-9b68-7089fa821e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.834759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6eb5b-a50c-411c-9b68-7089fa821e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.835743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6eb5b-a50c-411c-9b68-7089fa821e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.837686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6eb5b-a50c-411c-9b68-7089fa821e27 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.865996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.870112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 918/1815] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:19.891307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d19a-c688-48ba-aa6f-70fa8c45bd94 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.896777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.897775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.898499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.898499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] PATH_INFO: `/v3/projects/7f52ecaf83e441dbb4fc6ff1a0f935cf/users/ed3568023ead4f41bfe628c070ebf00f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.918986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f52ecaf83e441dbb4fc6ff1a0f935cf, user_id=ed3568023ead4f41bfe628c070ebf00f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.921730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.971860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 898/1816] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:19 2022] PUT /identity/v3/projects/7f52ecaf83e441dbb4fc6ff1a0f935cf/users/ed3568023ead4f41bfe628c070ebf00f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:19.976748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbff0-8fb3-412b-8467-6abd09f7c61e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.977326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbff0-8fb3-412b-8467-6abd09f7c61e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.977837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbff0-8fb3-412b-8467-6abd09f7c61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.047864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d72dbff0-8fb3-412b-8467-6abd09f7c61e None None] MFA Rules not processed for user `ed3568023ead4f41bfe628c070ebf00f`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:20.049601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d72dbff0-8fb3-412b-8467-6abd09f7c61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:20.174150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 919/1817] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:20.186138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.188189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:20.279298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c6efa-dad7-43b4-8480-95ce9391f42c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.283694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.284388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.284696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.285429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.296248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.297731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.307995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 899/1818] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:21.266444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dbff0-8fb3-412b-8467-6abd09f7c61e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:21.278300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dbff0-8fb3-412b-8467-6abd09f7c61e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.283862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626899a-b3df-4280-9c36-84ab10edc4f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.284891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626899a-b3df-4280-9c36-84ab10edc4f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.285358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626899a-b3df-4280-9c36-84ab10edc4f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.285809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626899a-b3df-4280-9c36-84ab10edc4f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.367757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626899a-b3df-4280-9c36-84ab10edc4f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.370860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626899a-b3df-4280-9c36-84ab10edc4f0 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.396241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 920/1819] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:29.283250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.284718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:29.291873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.378518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0697-f40e-4820-b12e-46ae4b23d623 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.384008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.384975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.385399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.385900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] PATH_INFO: `/v3/users/b7c46d0590ee4389ac76c661baf4dbcf` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.388236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c46d0590ee4389ac76c661baf4dbcf)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.390330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.461951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c46d0590ee4389ac76c661baf4dbcf'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:29.472493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c46d0590ee4389ac76c661baf4dbcf'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:29.482856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c46d0590ee4389ac76c661baf4dbcf'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:29.483533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c46d0590ee4389ac76c661baf4dbcf'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:29.500761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c46d0590ee4389ac76c661baf4dbcf'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:29.523282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 900/1820] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:29 2022] DELETE /identity/v3/users/b7c46d0590ee4389ac76c661baf4dbcf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:29.527807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.528683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.529113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.553978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:29.555251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:29.590517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.635433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 921/1821] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:29.671480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.674410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:29.805493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93791b5c-7f31-46bf-8907-1701c662abc6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.809527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.810236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.810590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.810934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.842696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.844893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.862272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 901/1822] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:31.317804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.319324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:31.393136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8484f-111c-4a5b-9f25-d8331d2b19bd None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.397868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.398701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.399206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.399662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.402659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:31.463430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.465987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:31.489835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d1cdf-c6d0-44ca-8f00-b67f092de22d service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.501035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.501681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.502044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.502532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] PATH_INFO: `/v3/projects/caa481bfbf834e078254e3d45bfb6ba1` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.520411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa481bfbf834e078254e3d45bfb6ba1)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.522385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.523879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.525408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.539215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 922/1823] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:31.582607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:31.650285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa481bfbf834e078254e3d45bfb6ba1', 'request_id': 'req-914dd662-ecd0-4041-a89d-c978144aaec4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.657374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa481bfbf834e078254e3d45bfb6ba1', 'request_id': 'req-914dd662-ecd0-4041-a89d-c978144aaec4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.665903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa481bfbf834e078254e3d45bfb6ba1', 'request_id': 'req-914dd662-ecd0-4041-a89d-c978144aaec4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.692890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] The token cache is being invalidate because project caa481bfbf834e078254e3d45bfb6ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:31.693765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.694672 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.696806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 902/1824] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:31 2022] DELETE /identity/v3/projects/caa481bfbf834e078254e3d45bfb6ba1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:31.701498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.714557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98404f-4391-4f93-860a-78185e4498ea service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.784296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.784947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.785231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.785502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] PATH_INFO: `/v3/users/509faa1cbc874eeb950ae1067c922ca6` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.786672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=509faa1cbc874eeb950ae1067c922ca6)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.788818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.830149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:31.855722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509faa1cbc874eeb950ae1067c922ca6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.857636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509faa1cbc874eeb950ae1067c922ca6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.869645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509faa1cbc874eeb950ae1067c922ca6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.871185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509faa1cbc874eeb950ae1067c922ca6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.890914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bdd4a859-4089-42e9-9ed0-e08878d82eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509faa1cbc874eeb950ae1067c922ca6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:31.905872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 923/1825] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:31 2022] DELETE /identity/v3/users/509faa1cbc874eeb950ae1067c922ca6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:32.246481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.248404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:32.300524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd662-ecd0-4041-a89d-c978144aaec4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.407382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.408604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.409106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.409606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] PATH_INFO: `/v3/projects/1b933318ecbe42818c3e91eadd15cd8d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.411044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b933318ecbe42818c3e91eadd15cd8d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.413293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.515520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b933318ecbe42818c3e91eadd15cd8d', 'request_id': 'req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:32.525993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b933318ecbe42818c3e91eadd15cd8d', 'request_id': 'req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:32.535768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b933318ecbe42818c3e91eadd15cd8d', 'request_id': 'req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:32.569100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] The token cache is being invalidate because project 1b933318ecbe42818c3e91eadd15cd8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:32.569503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:32.570297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9a62f8c-f043-4db6-84ae-220bc8a64cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:32.572374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 903/1826] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:32 2022] DELETE /identity/v3/projects/1b933318ecbe42818c3e91eadd15cd8d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:32.578000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7c132-2b72-4a6f-9911-87a98895a949 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.579030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7c132-2b72-4a6f-9911-87a98895a949 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.579484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7c132-2b72-4a6f-9911-87a98895a949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.641540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6be7c132-2b72-4a6f-9911-87a98895a949 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:32.644892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6be7c132-2b72-4a6f-9911-87a98895a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:32.737916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 924/1827] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:32.745936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ccfba-7d58-4565-8a51-d65bdcd35c16 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.747391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ccfba-7d58-4565-8a51-d65bdcd35c16 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.747768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ccfba-7d58-4565-8a51-d65bdcd35c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.786791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-821ccfba-7d58-4565-8a51-d65bdcd35c16 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:32.789105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-821ccfba-7d58-4565-8a51-d65bdcd35c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:32.824232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 904/1828] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:55:32.825374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7c132-2b72-4a6f-9911-87a98895a949 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.836572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7c132-2b72-4a6f-9911-87a98895a949 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.841979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.842597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.842875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.843148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.843799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:32.844100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.845949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.859891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 925/1829] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:32.865206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ccfba-7d58-4565-8a51-d65bdcd35c16 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.875178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ccfba-7d58-4565-8a51-d65bdcd35c16 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.880506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.881164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.881402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.881857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.882489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.884564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.920529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 905/1830] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:32 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:55:32.926191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.941535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bebf9-0290-401f-b604-a5441741f61f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.019041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.019778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.020969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.021575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.022472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.024955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.054597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 926/1831] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:32 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:33.059856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.073856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be556a-06ac-4c6a-95e6-fc49ee59554c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.082079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.083132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.083687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.083993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.084900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.086746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.093794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.102916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 906/1832] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:33.104212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.117633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaae1d6-014c-4e82-be34-de1e3aa0eecb admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.124195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cc037-a010-4dbf-b7cf-d66d64e14c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.125129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cc037-a010-4dbf-b7cf-d66d64e14c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.125651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cc037-a010-4dbf-b7cf-d66d64e14c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.126921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cc037-a010-4dbf-b7cf-d66d64e14c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.127951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cc037-a010-4dbf-b7cf-d66d64e14c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.130156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cc037-a010-4dbf-b7cf-d66d64e14c95 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.145387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 927/1833] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:33.152180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.163456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f9f3d-4cf2-4434-a1ef-22798d4356e1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.172988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.173909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.174091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.174471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] PATH_INFO: `/v3/projects/423f2240736646c5823ec1f1f41e12bd/users/e92de12f2ccb4b2ba1089469d0ed3514/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.199865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423f2240736646c5823ec1f1f41e12bd, user_id=e92de12f2ccb4b2ba1089469d0ed3514, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.203448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.232225 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 907/1834] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:33 2022] PUT /identity/v3/projects/423f2240736646c5823ec1f1f41e12bd/users/e92de12f2ccb4b2ba1089469d0ed3514/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:33.234342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.234634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.234909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.278344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.282135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] MFA Rules not processed for user `e92de12f2ccb4b2ba1089469d0ed3514`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:33.283706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:33.390345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 928/1835] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:33.402802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.405074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:33.497575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f48761-e965-4d6a-96f2-a0f7756879b4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.502271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8ba16-b6c0-469f-b5de-f80e68abff62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.502998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8ba16-b6c0-469f-b5de-f80e68abff62 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.503788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8ba16-b6c0-469f-b5de-f80e68abff62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.504938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8ba16-b6c0-469f-b5de-f80e68abff62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.519590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8ba16-b6c0-469f-b5de-f80e68abff62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.522114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8ba16-b6c0-469f-b5de-f80e68abff62 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.538450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 908/1836] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:34.576964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.578462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:34.652993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0900438-5ac4-45a6-aa7d-b4ee7fcef334 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.658310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.658955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.659331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.659499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] PATH_INFO: `/v3/users/9f800aac23ce487a8c95a0f64898059a` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.660956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f800aac23ce487a8c95a0f64898059a)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.662558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.742117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f800aac23ce487a8c95a0f64898059a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:34.742862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f800aac23ce487a8c95a0f64898059a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:34.754967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f800aac23ce487a8c95a0f64898059a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:34.755598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f800aac23ce487a8c95a0f64898059a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:34.770614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-883ad894-e007-4fca-b477-c9e9b2d12f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f800aac23ce487a8c95a0f64898059a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:34.788648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 929/1837] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:34 2022] DELETE /identity/v3/users/9f800aac23ce487a8c95a0f64898059a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:34.794740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.795238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.795316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.809954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e410cf5-f98c-4d8e-89af-57955fd7cd1f None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.810154 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e410cf5-f98c-4d8e-89af-57955fd7cd1f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.810407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e410cf5-f98c-4d8e-89af-57955fd7cd1f None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.812006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 930/1838] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 16:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:34.815813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:34.817143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:34.819088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.819384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.819665 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.848869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:34.850123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:34.903093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:34.911445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 909/1839] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:55:34.921818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.923228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:34.962901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 931/1840] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 16:55:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:55:34.975568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.977018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:34.985826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed69ed9-aa61-49fa-bbb4-543025fe0fec None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.990293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.990618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175f224-e676-49b5-9e51-58b452cb0649 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.990921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.991254 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.991473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.998447 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.999063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.999408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.999615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.003515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.005500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.008637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.010226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.019137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 910/1841] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:55:35.020742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 932/1842] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:36.080353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:36.082030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:36.099147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd018f6-12d0-4516-ae9e-aa035837ddf6 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:36.114415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:36.115259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:36.115766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:36.115967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] PATH_INFO: `/v3/projects/16c708ab63b243fea7d212f3a6a2d875` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:36.118783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c708ab63b243fea7d212f3a6a2d875)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:36.124197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:36.152757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:36.194232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c708ab63b243fea7d212f3a6a2d875', 'request_id': 'req-cfede9d7-8538-42a8-ad56-e51c98d482f9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.201933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c708ab63b243fea7d212f3a6a2d875', 'request_id': 'req-cfede9d7-8538-42a8-ad56-e51c98d482f9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.209056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c708ab63b243fea7d212f3a6a2d875', 'request_id': 'req-cfede9d7-8538-42a8-ad56-e51c98d482f9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.224363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] The token cache is being invalidate because project 16c708ab63b243fea7d212f3a6a2d875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:36.224778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.225437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.227286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 911/1843] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:36 2022] DELETE /identity/v3/projects/16c708ab63b243fea7d212f3a6a2d875 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:36.231542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:36.242218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b261-c5c5-4faa-8676-64a13ee5d368 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:36.308344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:36.309212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:36.309573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:36.309917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] PATH_INFO: `/v3/users/500ad182424b477181da6a11945301e7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:36.311102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=500ad182424b477181da6a11945301e7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:36.313124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:36.359469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500ad182424b477181da6a11945301e7'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.360270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500ad182424b477181da6a11945301e7'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.377149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500ad182424b477181da6a11945301e7'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.377760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500ad182424b477181da6a11945301e7'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.390858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500ad182424b477181da6a11945301e7'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.403814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 933/1844] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:36 2022] DELETE /identity/v3/users/500ad182424b477181da6a11945301e7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:36.686683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:36.688263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:36.703893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede9d7-8538-42a8-ad56-e51c98d482f9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:36.762386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:36.762951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:36.763228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:36.763502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] PATH_INFO: `/v3/projects/48a00db9285e478fbea16647ecb77663` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:36.764646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a00db9285e478fbea16647ecb77663)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:36.766567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:36.812703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a00db9285e478fbea16647ecb77663', 'request_id': 'req-a692b3aa-873b-471a-af81-41b2d34edcd2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.818421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a00db9285e478fbea16647ecb77663', 'request_id': 'req-a692b3aa-873b-471a-af81-41b2d34edcd2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.825334 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a00db9285e478fbea16647ecb77663', 'request_id': 'req-a692b3aa-873b-471a-af81-41b2d34edcd2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.840435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] The token cache is being invalidate because project 48a00db9285e478fbea16647ecb77663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:36.841071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.841708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.843571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 912/1845] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:36 2022] DELETE /identity/v3/projects/48a00db9285e478fbea16647ecb77663 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:36.848420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:36.850368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:36.870376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6061-0190-40f0-b507-2b8f21412dda admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:36.888685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:36.930796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:36.931415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:36.931690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:36.931960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] PATH_INFO: `/v3/users/ed3568023ead4f41bfe628c070ebf00f` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:36.933869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed3568023ead4f41bfe628c070ebf00f)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:36.935817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:36.984310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed3568023ead4f41bfe628c070ebf00f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.984962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed3568023ead4f41bfe628c070ebf00f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.993388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed3568023ead4f41bfe628c070ebf00f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:36.994003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed3568023ead4f41bfe628c070ebf00f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:37.007291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fc957a46-ec5f-4011-8f9e-04719d057aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed3568023ead4f41bfe628c070ebf00f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:37.020879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 934/1846] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:55:36 2022] DELETE /identity/v3/users/ed3568023ead4f41bfe628c070ebf00f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:55:37.384566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.386899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:37.402757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692b3aa-873b-471a-af81-41b2d34edcd2 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.471250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:37.503200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.504031 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.504432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.504772 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] PATH_INFO: `/v3/projects/7f52ecaf83e441dbb4fc6ff1a0f935cf` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.506128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f52ecaf83e441dbb4fc6ff1a0f935cf)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.508270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.561348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f52ecaf83e441dbb4fc6ff1a0f935cf', 'request_id': 'req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:37.567077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f52ecaf83e441dbb4fc6ff1a0f935cf', 'request_id': 'req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:37.574203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f52ecaf83e441dbb4fc6ff1a0f935cf', 'request_id': 'req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:37.592075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] The token cache is being invalidate because project 7f52ecaf83e441dbb4fc6ff1a0f935cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:55:37.592506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:37.593407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e56ad7dd-6f3f-483c-b34f-e4e0f96794b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:55:37.595690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 913/1847] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:55:37 2022] DELETE /identity/v3/projects/7f52ecaf83e441dbb4fc6ff1a0f935cf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:37.606231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705831b7-fd2a-410f-a512-c8cba111bff7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.606688 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705831b7-fd2a-410f-a512-c8cba111bff7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.606919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705831b7-fd2a-410f-a512-c8cba111bff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.625954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-705831b7-fd2a-410f-a512-c8cba111bff7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:37.627147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-705831b7-fd2a-410f-a512-c8cba111bff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:37.684565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 935/1848] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:55:37.692788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf752f0-9471-4d52-a36c-56d44b12122a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.693110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf752f0-9471-4d52-a36c-56d44b12122a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.693381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf752f0-9471-4d52-a36c-56d44b12122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.713830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-eaf752f0-9471-4d52-a36c-56d44b12122a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:37.715106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-eaf752f0-9471-4d52-a36c-56d44b12122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:37.723597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 914/1849] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:37.728018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705831b7-fd2a-410f-a512-c8cba111bff7 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.737745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705831b7-fd2a-410f-a512-c8cba111bff7 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.742276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.742874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.743157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.743430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.744077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:37.744414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.746032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.757168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 936/1850] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:37.761711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf752f0-9471-4d52-a36c-56d44b12122a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.770830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf752f0-9471-4d52-a36c-56d44b12122a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.775527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.776945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.777198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.777446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.778077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.779979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.806819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 915/1851] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:37 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:37.811060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.820969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf0a39-1e6a-40dd-9814-deb405d22712 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.846371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:37.880626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.881242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.881525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.881801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.882680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.884716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.908216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 937/1852] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:37 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:55:37.912320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.922560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c614bb-bcd7-4f57-9972-4284294c091c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.927157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.927763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.928043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.928357 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.929004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.930588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.941249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 916/1853] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:37.945098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.955438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af658-5386-4575-b54a-d05d2ca2cf12 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.960359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052229f0-c330-4b9e-8df8-66739fd5571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.960961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052229f0-c330-4b9e-8df8-66739fd5571a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.961246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052229f0-c330-4b9e-8df8-66739fd5571a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.961523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052229f0-c330-4b9e-8df8-66739fd5571a admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.962164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052229f0-c330-4b9e-8df8-66739fd5571a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.964410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052229f0-c330-4b9e-8df8-66739fd5571a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.974075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 938/1854] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:37.978153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.988073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee7ee0-722f-4400-81d3-8cc24b5b8206 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.992626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.993668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.993956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.994231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] PATH_INFO: `/v3/projects/527ee8944b054c068cbf0012b424b6b6/users/fa6d760143f8418f8186097358a3e3f0/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.008464 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=527ee8944b054c068cbf0012b424b6b6, user_id=fa6d760143f8418f8186097358a3e3f0, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.011643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.024167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 917/1855] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:37 2022] PUT /identity/v3/projects/527ee8944b054c068cbf0012b424b6b6/users/fa6d760143f8418f8186097358a3e3f0/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:38.035924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f63e6-b0a2-4c73-9849-f4c0486d200a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.036346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f63e6-b0a2-4c73-9849-f4c0486d200a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.036627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f63e6-b0a2-4c73-9849-f4c0486d200a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.064942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-201f63e6-b0a2-4c73-9849-f4c0486d200a None None] MFA Rules not processed for user `fa6d760143f8418f8186097358a3e3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:38.066064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-201f63e6-b0a2-4c73-9849-f4c0486d200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:38.130532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 939/1856] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:38.135164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.199166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501dea61-50e8-44f9-a76a-fddf5e04c0d7 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.203937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.204596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.204872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.205174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.205809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.207989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.213718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.238510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 918/1857] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:55:38 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:38.242727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f63e6-b0a2-4c73-9849-f4c0486d200a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.253861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f63e6-b0a2-4c73-9849-f4c0486d200a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.310350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.310975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.311260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.311538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.312213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.314135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.341022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 940/1858] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:38 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:55:38.341601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.352115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6d232-1310-4f48-bc46-24a63ecffa66 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.356992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.357595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.357873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.358142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.358782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.360411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.372220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 919/1859] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:38.376240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.391440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e5cc-9a73-4a6a-9fbf-c3394795fa74 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.396777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.398465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.398755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.399036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] PATH_INFO: `/v3/projects/65564b21768c4e8288cc5b41c633c80b/users/8581915903c04a84bc72548910afb303/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.427837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65564b21768c4e8288cc5b41c633c80b, user_id=8581915903c04a84bc72548910afb303, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.430790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.442658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.449077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 941/1860] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:38 2022] PUT /identity/v3/projects/65564b21768c4e8288cc5b41c633c80b/users/8581915903c04a84bc72548910afb303/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:55:38.453004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.469653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9b76c-09d4-4f42-91bf-e5b44dfe1010 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.566010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a21021-8d50-4c63-a59d-46a4dc6b2e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.567066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a21021-8d50-4c63-a59d-46a4dc6b2e11 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.567900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a21021-8d50-4c63-a59d-46a4dc6b2e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.568346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a21021-8d50-4c63-a59d-46a4dc6b2e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.569134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a21021-8d50-4c63-a59d-46a4dc6b2e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.570840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a21021-8d50-4c63-a59d-46a4dc6b2e11 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.583066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 920/1861] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:38.587158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.612377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb2560-98cc-4dcd-8a54-1c35fd4bc243 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.630041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.630906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.631263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.631606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] PATH_INFO: `/v3/projects/65564b21768c4e8288cc5b41c633c80b/users/8581915903c04a84bc72548910afb303/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.633865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] RBAC: Authorizing `identity:create_grant(project_id=65564b21768c4e8288cc5b41c633c80b, user_id=8581915903c04a84bc72548910afb303, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.640945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.648043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 942/1862] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:55:38 2022] PUT /identity/v3/projects/65564b21768c4e8288cc5b41c633c80b/users/8581915903c04a84bc72548910afb303/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:38.659066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1c7c7-08e5-4d8a-ae28-f24b3210e5e0 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.659727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1c7c7-08e5-4d8a-ae28-f24b3210e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.660311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1c7c7-08e5-4d8a-ae28-f24b3210e5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.697226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-82e1c7c7-08e5-4d8a-ae28-f24b3210e5e0 None None] MFA Rules not processed for user `8581915903c04a84bc72548910afb303`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:38.698490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-82e1c7c7-08e5-4d8a-ae28-f24b3210e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:38.785398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 921/1863] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:55:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:55:38.797170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.798873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:38.856526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ca813-eca3-4e4d-a5b0-225c41b6973b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.860093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.860734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.861016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.861293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.917530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.918992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.927800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 943/1864] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:48.782557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=73,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:48.924232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=53,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:49.743502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1c7c7-08e5-4d8a-ae28-f24b3210e5e0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.759113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1c7c7-08e5-4d8a-ae28-f24b3210e5e0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:49.763959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454d8a-621b-4640-ad69-42f2eeeb326a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:49.764841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454d8a-621b-4640-ad69-42f2eeeb326a service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.765313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454d8a-621b-4640-ad69-42f2eeeb326a service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.765577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454d8a-621b-4640-ad69-42f2eeeb326a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.777053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454d8a-621b-4640-ad69-42f2eeeb326a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:49.778955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454d8a-621b-4640-ad69-42f2eeeb326a service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:49.791509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 922/1865] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:55:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:55:59.785485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:08.581862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.660250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15955fac-d500-4307-a670-8b3c6aa6c38d service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.667438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.668401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.668804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.669722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] PATH_INFO: `/v3/users/fa6d760143f8418f8186097358a3e3f0` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.671035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6d760143f8418f8186097358a3e3f0)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.673385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.732472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6d760143f8418f8186097358a3e3f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:08.733013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6d760143f8418f8186097358a3e3f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:08.740927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6d760143f8418f8186097358a3e3f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:08.741531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6d760143f8418f8186097358a3e3f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:08.755085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6d760143f8418f8186097358a3e3f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:08.768984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 944/1866] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:08 2022] DELETE /identity/v3/users/fa6d760143f8418f8186097358a3e3f0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:08.773453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.773873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.774086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.793891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:08.795083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:08.853172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 923/1867] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:08.863643 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.865084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:08.924617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215031d5-6def-4558-b140-61ef507bf720 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.928399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.929014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.929288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.929558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.938995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.940562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.952370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 945/1868] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:09.355287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.356903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:09.371834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631390e0-7e1b-45ce-beb2-4eedb43d873f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.376678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.377350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.377624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.377895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] PATH_INFO: `/v3/projects/527ee8944b054c068cbf0012b424b6b6` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.379012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=527ee8944b054c068cbf0012b424b6b6)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.380981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.449036 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '527ee8944b054c068cbf0012b424b6b6', 'request_id': 'req-0c382945-1ecc-49e0-942c-59a715f9dea5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.454872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '527ee8944b054c068cbf0012b424b6b6', 'request_id': 'req-0c382945-1ecc-49e0-942c-59a715f9dea5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.461569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '527ee8944b054c068cbf0012b424b6b6', 'request_id': 'req-0c382945-1ecc-49e0-942c-59a715f9dea5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.475573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] The token cache is being invalidate because project 527ee8944b054c068cbf0012b424b6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:09.475867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.476953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.481988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 924/1869] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:09 2022] DELETE /identity/v3/projects/527ee8944b054c068cbf0012b424b6b6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:09.483326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.495080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb02f1-b221-497b-9ccc-08677979b565 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.516719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.568817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.569900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.570413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.570819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] PATH_INFO: `/v3/users/8581915903c04a84bc72548910afb303` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.573058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8581915903c04a84bc72548910afb303)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.575893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.690573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8581915903c04a84bc72548910afb303'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.691937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8581915903c04a84bc72548910afb303'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.711304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8581915903c04a84bc72548910afb303'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.712063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8581915903c04a84bc72548910afb303'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.725980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9949208e-4bd1-44cc-a3c5-247e3c5dd78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8581915903c04a84bc72548910afb303'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.738972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 946/1870] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:09 2022] DELETE /identity/v3/users/8581915903c04a84bc72548910afb303 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:09.743553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.745533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:09.758179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c382945-1ecc-49e0-942c-59a715f9dea5 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.840305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.841080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.841306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.841813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] PATH_INFO: `/v3/users/e92de12f2ccb4b2ba1089469d0ed3514` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.843155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92de12f2ccb4b2ba1089469d0ed3514)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.845171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.876766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.899816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92de12f2ccb4b2ba1089469d0ed3514'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.908761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92de12f2ccb4b2ba1089469d0ed3514'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.919815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92de12f2ccb4b2ba1089469d0ed3514'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.920538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92de12f2ccb4b2ba1089469d0ed3514'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.936208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92de12f2ccb4b2ba1089469d0ed3514'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:09.947808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 925/1871] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:09 2022] DELETE /identity/v3/users/e92de12f2ccb4b2ba1089469d0ed3514 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:09.952415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.952708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.952988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.975362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:09.976627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.050179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 947/1872] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:56:10.061429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.062836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.130816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.132530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.148200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d6d20-4ab5-4296-9c11-5172d771c8a6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.152497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbd938-c164-439e-8ca3-e912169eaef0 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.156406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178c746-9ac7-4149-98d7-888d57d67528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.157584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178c746-9ac7-4149-98d7-888d57d67528 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.157584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178c746-9ac7-4149-98d7-888d57d67528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.157584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178c746-9ac7-4149-98d7-888d57d67528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.159357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.160265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.160552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.160928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] PATH_INFO: `/v3/projects/65564b21768c4e8288cc5b41c633c80b` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.162559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65564b21768c4e8288cc5b41c633c80b)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.166712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.168259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178c746-9ac7-4149-98d7-888d57d67528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.169996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178c746-9ac7-4149-98d7-888d57d67528 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.180094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 926/1873] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:10.206671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:10.220076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65564b21768c4e8288cc5b41c633c80b', 'request_id': 'req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.227150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65564b21768c4e8288cc5b41c633c80b', 'request_id': 'req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.233840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65564b21768c4e8288cc5b41c633c80b', 'request_id': 'req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.254338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] The token cache is being invalidate because project 65564b21768c4e8288cc5b41c633c80b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:10.254642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.255320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c7fe2af4-63a9-4197-80bb-a1b60d1ce0d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.262085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 948/1874] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:10 2022] DELETE /identity/v3/projects/65564b21768c4e8288cc5b41c633c80b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:10.263591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0daf2d-8c9f-4653-88bc-b7b26cf99228 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.263875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0daf2d-8c9f-4653-88bc-b7b26cf99228 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.264177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0daf2d-8c9f-4653-88bc-b7b26cf99228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.285175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-db0daf2d-8c9f-4653-88bc-b7b26cf99228 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:10.286744 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-db0daf2d-8c9f-4653-88bc-b7b26cf99228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.353227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 927/1875] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:56:10.361292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861235d-ff27-4b5f-80a9-bdb2ae518e95 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.361585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861235d-ff27-4b5f-80a9-bdb2ae518e95 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.361865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861235d-ff27-4b5f-80a9-bdb2ae518e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.402491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e861235d-ff27-4b5f-80a9-bdb2ae518e95 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:10.404012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e861235d-ff27-4b5f-80a9-bdb2ae518e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.415599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 949/1876] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:10.419974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0daf2d-8c9f-4653-88bc-b7b26cf99228 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.430871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0daf2d-8c9f-4653-88bc-b7b26cf99228 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.438358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.439148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.439433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.440788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.440788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:10.441117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.444054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.459098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 928/1877] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:10.464328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861235d-ff27-4b5f-80a9-bdb2ae518e95 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.473620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861235d-ff27-4b5f-80a9-bdb2ae518e95 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.479464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.480072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.480387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.480659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.481283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.483523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.514474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 950/1878] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:10 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:10.515665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.526607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0b001-0ad3-4531-bf48-35048548f7a2 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.557724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:10.592618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.593255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.593534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.593952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.594538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.596465 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.614530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.616884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.621670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 929/1879] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:10 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:10.625747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.631268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe122c0-0c0d-45a2-8a05-b39581ccde5c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.636870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893700b-4ee5-4f94-8275-b1937e551a5c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.638318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.639031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.639383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.639753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] PATH_INFO: `/v3/projects/423f2240736646c5823ec1f1f41e12bd` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.641002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=423f2240736646c5823ec1f1f41e12bd)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.643019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.644485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.645224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.645556 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.645840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.646489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.648289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.664460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 930/1880] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:10.665749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.677718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a36ad-1ab9-4347-bb22-2acc7309b578 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.682729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.683285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.683621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.683825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.684523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.686086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.702562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423f2240736646c5823ec1f1f41e12bd', 'request_id': 'req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.709623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 931/1881] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:10.713583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423f2240736646c5823ec1f1f41e12bd', 'request_id': 'req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.714338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:10.723767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423f2240736646c5823ec1f1f41e12bd', 'request_id': 'req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.741138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf966925-cd12-4bdf-a694-dd6bfb82cff6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:10.741820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] The token cache is being invalidate because project 423f2240736646c5823ec1f1f41e12bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:10.742456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.743395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-9d9fac11-c23a-4f1f-98fe-0d6b1f1bb8cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:10.749496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 951/1882] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:10 2022] DELETE /identity/v3/projects/423f2240736646c5823ec1f1f41e12bd => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:10.756289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0df6-3c24-4b8e-8418-5f88f70db56e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.758161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0df6-3c24-4b8e-8418-5f88f70db56e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.758682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0df6-3c24-4b8e-8418-5f88f70db56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.804088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d6de0df6-3c24-4b8e-8418-5f88f70db56e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:10.805466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d6de0df6-3c24-4b8e-8418-5f88f70db56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.821373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790af627-1810-403d-9f35-12ddc10f8fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:10.822361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790af627-1810-403d-9f35-12ddc10f8fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.822648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790af627-1810-403d-9f35-12ddc10f8fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.823021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790af627-1810-403d-9f35-12ddc10f8fc7 admin admin] PATH_INFO: `/v3/projects/9cb3cf64930b4d7c82c2905b8a790079/users/940458c3248b4a60bcbd37cccc117082/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.844068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790af627-1810-403d-9f35-12ddc10f8fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb3cf64930b4d7c82c2905b8a790079, user_id=940458c3248b4a60bcbd37cccc117082, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:10.847067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790af627-1810-403d-9f35-12ddc10f8fc7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:10.867264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 932/1883] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:10 2022] PUT /identity/v3/projects/9cb3cf64930b4d7c82c2905b8a790079/users/940458c3248b4a60bcbd37cccc117082/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:10.880058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953303-4431-42e4-a99e-be47354d97ec None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.880408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953303-4431-42e4-a99e-be47354d97ec None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.880745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953303-4431-42e4-a99e-be47354d97ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.895970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 952/1884] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:10.904472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:10.904849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:10.905210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:10.924938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-52953303-4431-42e4-a99e-be47354d97ec None None] MFA Rules not processed for user `940458c3248b4a60bcbd37cccc117082`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:10.926039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:10.926250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-52953303-4431-42e4-a99e-be47354d97ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.927000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:10.944526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:11.025985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 953/1886] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:56:11.026198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 933/1886] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:56:11.027013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.029745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:11.030805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52953303-4431-42e4-a99e-be47354d97ec None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.045877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52953303-4431-42e4-a99e-be47354d97ec None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.049123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f79e58-c037-4540-bd5f-0b7b78408686 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.051988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.052786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.053093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.053368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.054152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:11.054555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.056347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.060069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.060942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.062077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.062880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.064269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.068346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.068471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:11.072564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 934/1887] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:11.075965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.099534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b24863-cfd5-4ffe-8db8-ddc05934b13c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.105287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.106706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.107280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.107485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.108617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.111669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.133955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 954/1888] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:11.138795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.150050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 935/1889] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:11.154704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.298793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e1d-6eda-47f6-9070-65509b7d3622 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.331676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ae3a3-2caa-43a1-af3b-7a11ba01339e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.338755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.339636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.340104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.340606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.341506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.343830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.372229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 936/1890] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:11.376095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.393881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56d2c4-3566-44a4-ba27-b5ca5dd5ed53 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.399214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.399913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.400329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.401304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.402191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.403928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.404988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.406510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.411751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.411751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.411751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.412845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.418762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 937/1891] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:11.422937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.437131 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fc8a6-0db4-47bc-a242-7e2e1a845c24 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.444208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 955/1892] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:11.445395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.445897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.446028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.446312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.446593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.447942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.449600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.459799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7892160-871c-4be9-8ccf-a5ba45251b29 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.461788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 938/1893] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:11.466573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.469978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.470609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.470882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.471145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.471799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.474083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.481572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eec7f-1a95-4de7-9307-d13132f31cb6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.488487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 956/1894] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:11.490303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c82354-1307-4345-abcf-ec642f74581e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.490984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c82354-1307-4345-abcf-ec642f74581e admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.491271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c82354-1307-4345-abcf-ec642f74581e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.491552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c82354-1307-4345-abcf-ec642f74581e admin admin] PATH_INFO: `/v3/projects/166eaa8341d043a1a10ae2f8c469cbb6/users/0b668ea99af44e1bab55eec2350e6b8c/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.493318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.504464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514876dd-8677-4de8-9c79-04a9020e8dbb admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.508226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c82354-1307-4345-abcf-ec642f74581e admin admin] RBAC: Authorizing `identity:create_grant(project_id=166eaa8341d043a1a10ae2f8c469cbb6, user_id=0b668ea99af44e1bab55eec2350e6b8c, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.509436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.510599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.510880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.511156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.511794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.512681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c82354-1307-4345-abcf-ec642f74581e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.513419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.524991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 957/1895] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:11.528928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 939/1896] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:11 2022] PUT /identity/v3/projects/166eaa8341d043a1a10ae2f8c469cbb6/users/0b668ea99af44e1bab55eec2350e6b8c/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:11.533812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.539114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f431e-c866-4b83-ad2a-1278066e7838 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.539409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f431e-c866-4b83-ad2a-1278066e7838 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.539679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f431e-c866-4b83-ad2a-1278066e7838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.542155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10e2a3-ed5b-4ece-ae24-748f3c0a86e8 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.555958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:11.568952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-148f431e-c866-4b83-ad2a-1278066e7838 None None] MFA Rules not processed for user `0b668ea99af44e1bab55eec2350e6b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:11.570193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-148f431e-c866-4b83-ad2a-1278066e7838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:11.619550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a77a6-f858-4c11-a5e8-90e891e08462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.620534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a77a6-f858-4c11-a5e8-90e891e08462 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.620883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a77a6-f858-4c11-a5e8-90e891e08462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.621527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a77a6-f858-4c11-a5e8-90e891e08462 admin admin] PATH_INFO: `/v3/projects/7e22fc5ed81f448480e2ad9766dfcc43/users/94ef93589f5942a4b8b8a584be122d34/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.641186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a77a6-f858-4c11-a5e8-90e891e08462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e22fc5ed81f448480e2ad9766dfcc43, user_id=94ef93589f5942a4b8b8a584be122d34, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.644039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a77a6-f858-4c11-a5e8-90e891e08462 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.661099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 958/1897] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:11 2022] PUT /identity/v3/projects/7e22fc5ed81f448480e2ad9766dfcc43/users/94ef93589f5942a4b8b8a584be122d34/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:11.666985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 940/1898] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:56:11.671700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b801394-91df-452f-9399-06dc59e762e6 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.672017 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b801394-91df-452f-9399-06dc59e762e6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.672298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b801394-91df-452f-9399-06dc59e762e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.677660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f431e-c866-4b83-ad2a-1278066e7838 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.690861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f431e-c866-4b83-ad2a-1278066e7838 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.699801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:11.701542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0b801394-91df-452f-9399-06dc59e762e6 None None] MFA Rules not processed for user `94ef93589f5942a4b8b8a584be122d34`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:11.702873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0b801394-91df-452f-9399-06dc59e762e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:11.777009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.777832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.778114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.778386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.779009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.780976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.792770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 959/1899] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:11.797637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b801394-91df-452f-9399-06dc59e762e6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.809753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b801394-91df-452f-9399-06dc59e762e6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.814481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.815222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 941/1900] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:11.816373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.816693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.816994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.817670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.819577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.823962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.847837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 960/1901] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:11.851902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.893631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bba06-9076-408a-bf7e-a55f387c79f6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.928558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f33526-be32-40ab-b141-582d22ffea7f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.934531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.935528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.936448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.936741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.937438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.939912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.950423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:11.958234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.959109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.959488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.959871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.960610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.962663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.972879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 961/1902] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/users => generated 298 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:11.974201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.987249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e7fc6-1978-4254-b8bf-2f775aa83ba2 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.989557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 942/1903] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:11.992430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.993541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.993705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.993988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.994299 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.995084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.996877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.005001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a1fb-c36c-4f4d-9c8c-d1da6d4f0e54 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.010091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 962/1904] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:12.011004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.011727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.012014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.012323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.013066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.014333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.014911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.027643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 943/1905] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:12.029560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0444739-e95e-4bfc-9ca3-ce34d88806ee admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.031432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.035126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.035995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.036475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.036900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] PATH_INFO: `/v3/projects/53c21d3dac6c460fa5260e74e52713f8/users/0b412b423ced46a48a0ff3b70453061b/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.048875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494248e-0e04-4f4e-bcbb-07cf7106ba0d admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.055821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.056491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c21d3dac6c460fa5260e74e52713f8, user_id=0b412b423ced46a48a0ff3b70453061b, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.056641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.056894 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.057172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.057818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.059417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.060764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.073399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 944/1906] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:12.077891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.085056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 963/1907] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:12 2022] PUT /identity/v3/projects/53c21d3dac6c460fa5260e74e52713f8/users/0b412b423ced46a48a0ff3b70453061b/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:12.085997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.097745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0e5e4-a124-4e58-aa52-fbdd7a081b03 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.110429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:12.189291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c64b1-9c73-4b65-96ca-d8a764c310c4 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.191689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959df93-f383-4aaa-afb7-dea7ae3f62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.192430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959df93-f383-4aaa-afb7-dea7ae3f62af admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.192714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959df93-f383-4aaa-afb7-dea7ae3f62af admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.192997 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959df93-f383-4aaa-afb7-dea7ae3f62af admin admin] PATH_INFO: `/v3/projects/1e7e4bb90b8e4fe294918388318daed3/users/16a3e640d3a04ea893688f328faedfba/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.195339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.196791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.197919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.198363 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.199449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.201514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.210852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959df93-f383-4aaa-afb7-dea7ae3f62af admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7e4bb90b8e4fe294918388318daed3, user_id=16a3e640d3a04ea893688f328faedfba, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.214944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 964/1908] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:12.215176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959df93-f383-4aaa-afb7-dea7ae3f62af admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.219395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.234022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb761d8-af53-4558-b579-481bfcab5a41 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.235569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 945/1909] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:12 2022] PUT /identity/v3/projects/1e7e4bb90b8e4fe294918388318daed3/users/16a3e640d3a04ea893688f328faedfba/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:12.249235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076bb79-264a-4b92-ae13-06e8eed87c9a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.249567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076bb79-264a-4b92-ae13-06e8eed87c9a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.249879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076bb79-264a-4b92-ae13-06e8eed87c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.283543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c076bb79-264a-4b92-ae13-06e8eed87c9a None None] MFA Rules not processed for user `16a3e640d3a04ea893688f328faedfba`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:12.284817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c076bb79-264a-4b92-ae13-06e8eed87c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:12.307659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce7c0d-503b-4bf0-a189-63a88e9fa2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.308856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7c0d-503b-4bf0-a189-63a88e9fa2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.309146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7c0d-503b-4bf0-a189-63a88e9fa2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.309423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7c0d-503b-4bf0-a189-63a88e9fa2cf admin admin] PATH_INFO: `/v3/projects/53c21d3dac6c460fa5260e74e52713f8/users/0b412b423ced46a48a0ff3b70453061b/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.313004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce7c0d-503b-4bf0-a189-63a88e9fa2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c21d3dac6c460fa5260e74e52713f8, user_id=0b412b423ced46a48a0ff3b70453061b, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.316605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce7c0d-503b-4bf0-a189-63a88e9fa2cf admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.335326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 965/1910] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:12 2022] PUT /identity/v3/projects/53c21d3dac6c460fa5260e74e52713f8/users/0b412b423ced46a48a0ff3b70453061b/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:12.347556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716b367-bf58-4a23-9a36-106d0c622c2f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.347869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716b367-bf58-4a23-9a36-106d0c622c2f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.348188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716b367-bf58-4a23-9a36-106d0c622c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.380747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 946/1911] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:12.381048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c716b367-bf58-4a23-9a36-106d0c622c2f None None] MFA Rules not processed for user `0b412b423ced46a48a0ff3b70453061b`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:12.382441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c716b367-bf58-4a23-9a36-106d0c622c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:12.392281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076bb79-264a-4b92-ae13-06e8eed87c9a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.404903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076bb79-264a-4b92-ae13-06e8eed87c9a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.448737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:12.484340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 966/1912] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:12.502669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.503299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.503578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.503852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.504521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.506424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.544349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 947/1913] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:12 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:12.547119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716b367-bf58-4a23-9a36-106d0c622c2f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.560186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716b367-bf58-4a23-9a36-106d0c622c2f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.636106 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.637355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.638237 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.638541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.639686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.642726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.672221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 967/1914] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:12 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:12.672694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.672694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:12.675204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.693526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:12.699627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00dacf-0ce1-496c-9952-b3cd8aa544ff admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.705147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.705753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.705987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.706848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.707083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.708991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.721905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 968/1915] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:12.727460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.742501 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c89a-b088-4062-bc6f-c5d871889d25 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.752025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.753197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.753593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.754483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] PATH_INFO: `/v3/projects/2e908bec76494639b66159464c78b4d1/users/82d0bd4e80094372a8d333a06b8d5c16/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.758742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9620e26-18a5-46a7-aa3f-152ca6efb0df admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.763260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.763876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.764179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.764455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.774627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e908bec76494639b66159464c78b4d1, user_id=82d0bd4e80094372a8d333a06b8d5c16, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.778236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.800717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 969/1916] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:12 2022] PUT /identity/v3/projects/2e908bec76494639b66159464c78b4d1/users/82d0bd4e80094372a8d333a06b8d5c16/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:12.805493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.844353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.845916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.858302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 948/1917] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:12.935056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ed06f-cbbd-4933-ad43-c0ef1ed36948 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.954710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9cbec-2a35-4508-98f8-46d389a623ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.955354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9cbec-2a35-4508-98f8-46d389a623ca admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.955634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9cbec-2a35-4508-98f8-46d389a623ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.955906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9cbec-2a35-4508-98f8-46d389a623ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.956641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9cbec-2a35-4508-98f8-46d389a623ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.958543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9cbec-2a35-4508-98f8-46d389a623ca admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.970452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 970/1918] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:12.974408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.990212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9ccef-42b8-4a94-98c9-44baf9153d71 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.995840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.996548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.996872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.997357 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] PATH_INFO: `/v3/projects/2e908bec76494639b66159464c78b4d1/users/82d0bd4e80094372a8d333a06b8d5c16/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:13.003524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e908bec76494639b66159464c78b4d1, user_id=82d0bd4e80094372a8d333a06b8d5c16, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:13.007090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:13.031004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 949/1919] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:12 2022] PUT /identity/v3/projects/2e908bec76494639b66159464c78b4d1/users/82d0bd4e80094372a8d333a06b8d5c16/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:13.041706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94775ae-fb84-42e2-9bf6-a4e449aac63f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:13.042162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94775ae-fb84-42e2-9bf6-a4e449aac63f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:13.042574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94775ae-fb84-42e2-9bf6-a4e449aac63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:13.075167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e94775ae-fb84-42e2-9bf6-a4e449aac63f None None] MFA Rules not processed for user `82d0bd4e80094372a8d333a06b8d5c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:13.076410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e94775ae-fb84-42e2-9bf6-a4e449aac63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:13.092081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:13.155880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 971/1920] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:13.172012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:13.173823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:13.270725 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc07fb-defb-4b3c-824e-bda5386fc58e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:13.274571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:13.275440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:13.275726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:13.276000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:13.337844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:13.339328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:13.348948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 950/1921] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:18.707275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94775ae-fb84-42e2-9bf6-a4e449aac63f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:18.721147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94775ae-fb84-42e2-9bf6-a4e449aac63f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:18.724937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:18.725568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:18.725854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:18.726139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:18.794681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:18.796334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:18.806212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 972/1922] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:19.116430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:19.123195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:19.127528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702cfd0-9659-4e13-a5dd-8c44f8a4f5ba service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:19.132846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:19.133879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:19.134340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:19.134676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:19.203403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:19.205078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:19.218358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 951/1923] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:19.549169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:19.557601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd47f1f-0ae3-46e2-ae76-d0fdda36dd2f service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:19.562137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:19.562774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:19.563054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:19.563324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:19.629426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:19.631215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:19.641328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 973/1924] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:19.849433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:19.859183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1061-be29-4192-a5df-a7ade3a06f05 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:19.863565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe3f7e-ac94-48c2-a448-300e4ad0c108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:19.864297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe3f7e-ac94-48c2-a448-300e4ad0c108 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:19.864643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe3f7e-ac94-48c2-a448-300e4ad0c108 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:19.864979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe3f7e-ac94-48c2-a448-300e4ad0c108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:19.877263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe3f7e-ac94-48c2-a448-300e4ad0c108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:19.879074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe3f7e-ac94-48c2-a448-300e4ad0c108 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:19.890854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 952/1925] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:21.651521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:21.653750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:21.699846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:21.720690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573bd96-c678-4fb6-bb07-04eab6fd8d07 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:21.725255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:21.725898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:21.726181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:21.726459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] PATH_INFO: `/v3/users/612336b0bfa64842ab15c0a9ba985dd4` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:21.727591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612336b0bfa64842ab15c0a9ba985dd4)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:21.729569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:21.790603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612336b0bfa64842ab15c0a9ba985dd4'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:21.792128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612336b0bfa64842ab15c0a9ba985dd4'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:21.802489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612336b0bfa64842ab15c0a9ba985dd4'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:21.803205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612336b0bfa64842ab15c0a9ba985dd4'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:21.819631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612336b0bfa64842ab15c0a9ba985dd4'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:21.832852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 974/1926] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:21 2022] DELETE /identity/v3/users/612336b0bfa64842ab15c0a9ba985dd4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:21.837477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1111722-a50d-447d-ac58-f2327018651a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:21.837943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1111722-a50d-447d-ac58-f2327018651a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:21.838043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1111722-a50d-447d-ac58-f2327018651a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:21.858026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f1111722-a50d-447d-ac58-f2327018651a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:21.859285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f1111722-a50d-447d-ac58-f2327018651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:21.925410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 953/1927] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:21.935809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:21.937592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:22.004548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e28f44-af4e-4a94-bc2e-69147baa9161 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:22.008566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:22.009231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:22.009512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:22.009785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:22.021735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:22.023326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:22.035755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 975/1928] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:22.416730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1111722-a50d-447d-ac58-f2327018651a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:22.418444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f1111722-a50d-447d-ac58-f2327018651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:22.460240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1111722-a50d-447d-ac58-f2327018651a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:22.466992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:22.467743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:22.468139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:22.468751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] PATH_INFO: `/v3/projects/02d19faf494248a08030749766b5d633` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:22.470007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d19faf494248a08030749766b5d633)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:22.472252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:22.550937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d19faf494248a08030749766b5d633', 'request_id': 'req-b7e33ae7-0604-4888-89f4-b7a020e5df3f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.559090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d19faf494248a08030749766b5d633', 'request_id': 'req-b7e33ae7-0604-4888-89f4-b7a020e5df3f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.570787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d19faf494248a08030749766b5d633', 'request_id': 'req-b7e33ae7-0604-4888-89f4-b7a020e5df3f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.598676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] The token cache is being invalidate because project 02d19faf494248a08030749766b5d633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:22.598982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.599988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.607155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 954/1929] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:22 2022] DELETE /identity/v3/projects/02d19faf494248a08030749766b5d633 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:22.611786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:22.629828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e04836-ac5f-496b-be5f-360884677d49 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:22.708970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:22.709633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:22.710044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:22.710385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] PATH_INFO: `/v3/users/ccc17e1851364e3ca24e593b2c0ff46e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:22.711649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc17e1851364e3ca24e593b2c0ff46e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:22.714022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:22.721467 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:22.789724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc17e1851364e3ca24e593b2c0ff46e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.790391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc17e1851364e3ca24e593b2c0ff46e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.799604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc17e1851364e3ca24e593b2c0ff46e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.800298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc17e1851364e3ca24e593b2c0ff46e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.812591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-531361d7-793e-4fbb-99ff-1bb7d988476c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc17e1851364e3ca24e593b2c0ff46e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:22.825634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 976/1930] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:22 2022] DELETE /identity/v3/users/ccc17e1851364e3ca24e593b2c0ff46e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:23.108694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.110150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:23.125146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e33ae7-0604-4888-89f4-b7a020e5df3f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.137661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:23.191091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.191841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.192127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.192441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] PATH_INFO: `/v3/projects/672652cb49c1427ab2f9396439b238a5` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.195708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672652cb49c1427ab2f9396439b238a5)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.197721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.256043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672652cb49c1427ab2f9396439b238a5', 'request_id': 'req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.263515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672652cb49c1427ab2f9396439b238a5', 'request_id': 'req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.269896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672652cb49c1427ab2f9396439b238a5', 'request_id': 'req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.284716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] The token cache is being invalidate because project 672652cb49c1427ab2f9396439b238a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:23.285596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.286247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-2fd64ce0-ccdc-4cdb-b510-0d712e313db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.287795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 955/1931] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:23 2022] DELETE /identity/v3/projects/672652cb49c1427ab2f9396439b238a5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:23.292910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab54e6b-afc0-4478-a675-b1d0a7ee8a72 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.293227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab54e6b-afc0-4478-a675-b1d0a7ee8a72 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.293541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab54e6b-afc0-4478-a675-b1d0a7ee8a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.314032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-eab54e6b-afc0-4478-a675-b1d0a7ee8a72 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:23.315256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-eab54e6b-afc0-4478-a675-b1d0a7ee8a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:23.376713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 977/1932] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:23.384472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1d8f-0b55-430e-899c-4076aa34d5e4 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.384764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1d8f-0b55-430e-899c-4076aa34d5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.385037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1d8f-0b55-430e-899c-4076aa34d5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.404437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a04a1d8f-0b55-430e-899c-4076aa34d5e4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:23.405662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a04a1d8f-0b55-430e-899c-4076aa34d5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:23.414459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 956/1933] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:23.419289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab54e6b-afc0-4478-a675-b1d0a7ee8a72 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.429294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab54e6b-afc0-4478-a675-b1d0a7ee8a72 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.435220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.435845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.436310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.436589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.437247 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:23.437549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.439261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.455756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 978/1934] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:23.456783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1d8f-0b55-430e-899c-4076aa34d5e4 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.466582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1d8f-0b55-430e-899c-4076aa34d5e4 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.471369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.471967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.472343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.472618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.473268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.475190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.510084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 957/1935] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:23 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:23.511191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.522406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe06a58-cc09-4405-9d76-f5a79823d13d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.607163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.607796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.608082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.608394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.616825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.621642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.643645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:23.655681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 979/1936] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:23 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:23.656912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.672466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7ef42-6029-43e9-bbe6-8d22d4268bcc admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.683035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.686781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.687176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.687692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.690442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.695113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.710220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 958/1937] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:23.715071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.726322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ced6b9-8c1c-4712-8740-3ca3a251e9d1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.732063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.732717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.732998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.733495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.734067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.735754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.760536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.760811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 980/1938] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:23.768712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.771569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1f43e-01e5-42d0-ae91-af3a24699eaa admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.773527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:23.776809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51491d1d-9d3b-4c33-91ca-5a6d0cb0322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.778295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51491d1d-9d3b-4c33-91ca-5a6d0cb0322d admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.778716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51491d1d-9d3b-4c33-91ca-5a6d0cb0322d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.779097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51491d1d-9d3b-4c33-91ca-5a6d0cb0322d admin admin] PATH_INFO: `/v3/projects/c214a60752ad408d9cdfd8abbd22013c/users/c0ec60ff5d2145f69c2ecb63c61bec4f/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.806323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d78e5a-9942-48b0-8898-064318fd7964 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.811576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.812344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.812694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.813036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] PATH_INFO: `/v3/users/940458c3248b4a60bcbd37cccc117082` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.814302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=940458c3248b4a60bcbd37cccc117082)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.816799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.821796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51491d1d-9d3b-4c33-91ca-5a6d0cb0322d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c214a60752ad408d9cdfd8abbd22013c, user_id=c0ec60ff5d2145f69c2ecb63c61bec4f, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.828718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51491d1d-9d3b-4c33-91ca-5a6d0cb0322d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.859595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 959/1939] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:23 2022] PUT /identity/v3/projects/c214a60752ad408d9cdfd8abbd22013c/users/c0ec60ff5d2145f69c2ecb63c61bec4f/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:23.871100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.871568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.871993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.887723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '940458c3248b4a60bcbd37cccc117082'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.888452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '940458c3248b4a60bcbd37cccc117082'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.899831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '940458c3248b4a60bcbd37cccc117082'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.900916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '940458c3248b4a60bcbd37cccc117082'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.916786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6a8979d9-ebee-4128-a8bc-e6c08e7e82ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '940458c3248b4a60bcbd37cccc117082'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:23.933268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 981/1940] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:23 2022] DELETE /identity/v3/users/940458c3248b4a60bcbd37cccc117082 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:23.939157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.940102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.940567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.949420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] MFA Rules not processed for user `c0ec60ff5d2145f69c2ecb63c61bec4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:23.951740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:23.963122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:23.965421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:24.017705 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:24.081562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 982/1941] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:24.081755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 960/1942] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:24.083167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.085414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:24.091568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.093223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:24.106704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb96ea-875c-4fce-8ac7-bd01c166ead9 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.114223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.114952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.115892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.116218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.117054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.119701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.165439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 961/1943] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:24 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:24.169389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.221537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a8809-5ec1-4a8c-982d-0aef0233848e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.261177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f23ca-6cb0-4501-9ca6-cc47df983112 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.266689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:24.267675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.268412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.268668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.268951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.270281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.273031 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.307617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 962/1944] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:24 2022] POST /identity/v3/users => generated 300 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:24.309835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.316160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.317146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.317543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.317907 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.320276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c14f6-3925-41ef-8092-a9c6a1b3b986 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.325132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.325736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.326019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.326637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.327064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.328898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.329329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.341039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.344207 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 963/1945] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:24.354020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 983/1946] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:24.355818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.378715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9851c5-5863-4291-be65-0f6594426168 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.385433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.386687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.387184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.387643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] PATH_INFO: `/v3/projects/34200e2b43b2453b81b096689df66555/users/0432301033b84c929bf674c7524e4562/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.416383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34200e2b43b2453b81b096689df66555, user_id=0432301033b84c929bf674c7524e4562, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.422691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.452265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 964/1947] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:24 2022] PUT /identity/v3/projects/34200e2b43b2453b81b096689df66555/users/0432301033b84c929bf674c7524e4562/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:24.457073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.473421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d172896-8536-437c-8d89-934b586a3ea6 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.573196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29de7ac-7d56-4f65-9671-eb8524ce8b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.575589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29de7ac-7d56-4f65-9671-eb8524ce8b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.575589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29de7ac-7d56-4f65-9671-eb8524ce8b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.578390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29de7ac-7d56-4f65-9671-eb8524ce8b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.579268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29de7ac-7d56-4f65-9671-eb8524ce8b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.584263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29de7ac-7d56-4f65-9671-eb8524ce8b58 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.613653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 984/1948] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:24.617549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.640061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4890434-9cad-47cd-a55c-af0701f32328 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.679187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.680348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.682004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.683758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] PATH_INFO: `/v3/projects/34200e2b43b2453b81b096689df66555/users/0432301033b84c929bf674c7524e4562/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.687937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34200e2b43b2453b81b096689df66555, user_id=0432301033b84c929bf674c7524e4562, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.691191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.720743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 965/1949] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:24 2022] PUT /identity/v3/projects/34200e2b43b2453b81b096689df66555/users/0432301033b84c929bf674c7524e4562/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:24.724613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.725090 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.725338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.780442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] MFA Rules not processed for user `0432301033b84c929bf674c7524e4562`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:24.786810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.788116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:24.789285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:24.875566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:24.899770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86da14-9f8c-4140-b657-27585243fc82 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.906258 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.907089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.907447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.907790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] PATH_INFO: `/v3/projects/9cb3cf64930b4d7c82c2905b8a790079` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.909166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb3cf64930b4d7c82c2905b8a790079)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.911447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.923934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 985/1950] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:24.939149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.942396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:24.969887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb3cf64930b4d7c82c2905b8a790079', 'request_id': 'req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:24.975957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb3cf64930b4d7c82c2905b8a790079', 'request_id': 'req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:24.984433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb3cf64930b4d7c82c2905b8a790079', 'request_id': 'req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.004307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] The token cache is being invalidate because project 9cb3cf64930b4d7c82c2905b8a790079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:25.004956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.007167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.007952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:25.008982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 966/1951] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:24 2022] DELETE /identity/v3/projects/9cb3cf64930b4d7c82c2905b8a790079 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:25.013885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:25.016213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.023701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053673a-467d-4f05-8e89-7bfd0c54330b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:25.094305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e55b8b-6ca0-4f69-9e8a-d0532e87fe20 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:25.097232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:25.098056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:25.098589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:25.099250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:25.100684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:25.101452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:25.101809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.101923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:25.102837 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] PATH_INFO: `/v3/users/94ef93589f5942a4b8b8a584be122d34` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:25.103950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ef93589f5942a4b8b8a584be122d34)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:25.107461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:25.180521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ef93589f5942a4b8b8a584be122d34'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.197088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ef93589f5942a4b8b8a584be122d34'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.222278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ef93589f5942a4b8b8a584be122d34'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.223045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ef93589f5942a4b8b8a584be122d34'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.230324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:25.234790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:25.245371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ef93589f5942a4b8b8a584be122d34'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.247773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:25.257052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 986/1952] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:56:25.259257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 967/1953] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:25 2022] DELETE /identity/v3/users/94ef93589f5942a4b8b8a584be122d34 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:25.338397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:25.340185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.371211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:25.373076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.373804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6121c7-3cde-481e-ae43-7e21e8af1258 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:25.523911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6fc3b-18f0-4289-8824-b440729d57ec admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:25.534985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:25.537355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:25.537551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:25.537993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:25.538124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:25.538405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:25.539023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] PATH_INFO: `/v3/users/c0ec60ff5d2145f69c2ecb63c61bec4f` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:25.540336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ec60ff5d2145f69c2ecb63c61bec4f)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:25.543797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:25.548877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] PATH_INFO: `/v3/users/0b668ea99af44e1bab55eec2350e6b8c` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:25.550427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b668ea99af44e1bab55eec2350e6b8c)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:25.562960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:25.638368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ec60ff5d2145f69c2ecb63c61bec4f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.648026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ec60ff5d2145f69c2ecb63c61bec4f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.661368 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b668ea99af44e1bab55eec2350e6b8c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.662346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b668ea99af44e1bab55eec2350e6b8c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.664866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ec60ff5d2145f69c2ecb63c61bec4f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.666646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ec60ff5d2145f69c2ecb63c61bec4f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.667001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:25.679219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b668ea99af44e1bab55eec2350e6b8c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.680120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b668ea99af44e1bab55eec2350e6b8c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.701419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-010dc83d-8a78-445c-8a48-0b5126a6c181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ec60ff5d2145f69c2ecb63c61bec4f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.711308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 968/1954] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:25 2022] DELETE /identity/v3/users/c0ec60ff5d2145f69c2ecb63c61bec4f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:25.714424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-32624b61-2673-469d-84cf-45a95636f4f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b668ea99af44e1bab55eec2350e6b8c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:25.718779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:25.724506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:25.731899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:25.749526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 987/1955] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:25 2022] DELETE /identity/v3/users/0b668ea99af44e1bab55eec2350e6b8c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:25.754900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:25.755997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:25.756475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:25.781564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:25.784955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.806648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:25.809058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.918395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 988/1956] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:25.919562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:25.938294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.942810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 969/1957] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:25.943678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:25.949013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:25.954972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc25db2-5a8a-4af8-b2d6-6248f2f6f1cb None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:25.961387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:25.962049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:25.962337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:25.963043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] PATH_INFO: `/v3/projects/7e22fc5ed81f448480e2ad9766dfcc43` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:25.964878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e22fc5ed81f448480e2ad9766dfcc43)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:25.966872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:25.994710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:26.032729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e22fc5ed81f448480e2ad9766dfcc43', 'request_id': 'req-476b1143-40a7-4a7b-926a-19007960459a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.044928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e22fc5ed81f448480e2ad9766dfcc43', 'request_id': 'req-476b1143-40a7-4a7b-926a-19007960459a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.054286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43150325-e4c3-4b7b-aab1-7010cc4a29aa None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:26.055254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e22fc5ed81f448480e2ad9766dfcc43', 'request_id': 'req-476b1143-40a7-4a7b-926a-19007960459a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.075761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] The token cache is being invalidate because project 7e22fc5ed81f448480e2ad9766dfcc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:26.076461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.077371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.079675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 989/1958] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:25 2022] DELETE /identity/v3/projects/7e22fc5ed81f448480e2ad9766dfcc43 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:26.080921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:26.082926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:26.134305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:26.134850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:26.135130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:26.135410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:26.137445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:26.175704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b1143-40a7-4a7b-926a-19007960459a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:26.180696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:26.181584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:26.182101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:26.182204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:26.184362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:26.193568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:26.264249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:26.266538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:26.289937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:26.291888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:26.297871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 970/1959] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:26.298890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:26.302771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:26.307834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 990/1960] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:26.319595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d3022-ef14-4bbb-a62c-cd262b06c5d3 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:26.333683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:26.334358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:26.334642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:26.334920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] PATH_INFO: `/v3/users/0b412b423ced46a48a0ff3b70453061b` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:26.336294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b412b423ced46a48a0ff3b70453061b)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:26.341537 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:26.491072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b412b423ced46a48a0ff3b70453061b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.500537 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b412b423ced46a48a0ff3b70453061b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.512713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b412b423ced46a48a0ff3b70453061b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.513542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b412b423ced46a48a0ff3b70453061b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.533804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:26.535359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b412b423ced46a48a0ff3b70453061b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:26.547627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 971/1961] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:26 2022] DELETE /identity/v3/users/0b412b423ced46a48a0ff3b70453061b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:27.165480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:27.168612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:27.191136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587255d-1726-458c-b439-c3337b02c3e2 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:27.224815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:27.226482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:27.330944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:27.331596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.331926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.332234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] PATH_INFO: `/v3/projects/c214a60752ad408d9cdfd8abbd22013c` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.334070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c214a60752ad408d9cdfd8abbd22013c)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:27.336383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:27.343226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183a576-969f-4663-8fc7-3321384c9496 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:27.351707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:27.352561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.352896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.353270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] PATH_INFO: `/v3/projects/53c21d3dac6c460fa5260e74e52713f8` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.354605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c21d3dac6c460fa5260e74e52713f8)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:27.357446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:27.430152 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c214a60752ad408d9cdfd8abbd22013c', 'request_id': 'req-7ba350df-e7b2-45bb-baea-e95389db1b2d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.438035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c214a60752ad408d9cdfd8abbd22013c', 'request_id': 'req-7ba350df-e7b2-45bb-baea-e95389db1b2d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.445322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c214a60752ad408d9cdfd8abbd22013c', 'request_id': 'req-7ba350df-e7b2-45bb-baea-e95389db1b2d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.448706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c21d3dac6c460fa5260e74e52713f8', 'request_id': 'req-71137673-74cb-4f1a-a067-7ece8892e3a8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.455891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c21d3dac6c460fa5260e74e52713f8', 'request_id': 'req-71137673-74cb-4f1a-a067-7ece8892e3a8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.460609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] The token cache is being invalidate because project c214a60752ad408d9cdfd8abbd22013c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:27.461004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.461713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.463192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c21d3dac6c460fa5260e74e52713f8', 'request_id': 'req-71137673-74cb-4f1a-a067-7ece8892e3a8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.467409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 991/1962] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:27 2022] DELETE /identity/v3/projects/c214a60752ad408d9cdfd8abbd22013c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:27.467409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:27.467409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:27.481807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] The token cache is being invalidate because project 53c21d3dac6c460fa5260e74e52713f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:27.482147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.482825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.487746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 972/1963] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:27 2022] DELETE /identity/v3/projects/53c21d3dac6c460fa5260e74e52713f8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:27.487944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:27.489542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:27.508644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:27.562766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba350df-e7b2-45bb-baea-e95389db1b2d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:27.586157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71137673-74cb-4f1a-a067-7ece8892e3a8 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:27.596882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:27.597545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.597826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.598106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] PATH_INFO: `/v3/users/0432301033b84c929bf674c7524e4562` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.599327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0432301033b84c929bf674c7524e4562)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:27.601353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:27.634430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:27.635410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.636404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.636527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:27.636623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] PATH_INFO: `/v3/projects/166eaa8341d043a1a10ae2f8c469cbb6` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.637621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=166eaa8341d043a1a10ae2f8c469cbb6)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:27.639933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:27.665520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0432301033b84c929bf674c7524e4562'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.673963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0432301033b84c929bf674c7524e4562'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.684843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0432301033b84c929bf674c7524e4562'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.685518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0432301033b84c929bf674c7524e4562'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.702376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1f528f17-97e0-46b4-95cc-cf4d99042f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0432301033b84c929bf674c7524e4562'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.710506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 973/1964] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:27 2022] DELETE /identity/v3/users/0432301033b84c929bf674c7524e4562 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:27.712178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafb3d8-3e4c-4241-bc3d-3245db03b3d7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.712552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafb3d8-3e4c-4241-bc3d-3245db03b3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.712998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafb3d8-3e4c-4241-bc3d-3245db03b3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.725893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166eaa8341d043a1a10ae2f8c469cbb6', 'request_id': 'req-e794c4af-43a2-4841-87d8-a9564a50a503'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.734214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166eaa8341d043a1a10ae2f8c469cbb6', 'request_id': 'req-e794c4af-43a2-4841-87d8-a9564a50a503'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.742164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166eaa8341d043a1a10ae2f8c469cbb6', 'request_id': 'req-e794c4af-43a2-4841-87d8-a9564a50a503'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.767348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-bdafb3d8-3e4c-4241-bc3d-3245db03b3d7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:27.770279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-bdafb3d8-3e4c-4241-bc3d-3245db03b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:27.771754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] The token cache is being invalidate because project 166eaa8341d043a1a10ae2f8c469cbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:27.772264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.773381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:27.777454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 992/1965] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:27 2022] DELETE /identity/v3/projects/166eaa8341d043a1a10ae2f8c469cbb6 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:27.780965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:27.782685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:27.854545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 974/1966] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:27.863018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee67e9-40e1-4e69-a26e-d979f0342590 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.863314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee67e9-40e1-4e69-a26e-d979f0342590 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.863590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee67e9-40e1-4e69-a26e-d979f0342590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.890839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5dee67e9-40e1-4e69-a26e-d979f0342590 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:27.892346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5dee67e9-40e1-4e69-a26e-d979f0342590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:27.896079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794c4af-43a2-4841-87d8-a9564a50a503 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:27.902480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 975/1967] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:27.907167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee67e9-40e1-4e69-a26e-d979f0342590 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:27.907624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:27.908510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.908794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.909065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] PATH_INFO: `/v3/users/16a3e640d3a04ea893688f328faedfba` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.911693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a3e640d3a04ea893688f328faedfba)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:27.914108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:27.921745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee67e9-40e1-4e69-a26e-d979f0342590 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:27.936745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:27.938161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:27.939152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:27.939439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:27.939712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:27.940416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:27.940723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:27.942410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:27.976785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 976/1968] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:27.981306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.002934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f0c4-2617-48fe-a329-a8b8d6748e7b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.020830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.021497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.021732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.022003 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.022637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.023203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a3e640d3a04ea893688f328faedfba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.024476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a3e640d3a04ea893688f328faedfba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.024628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.042036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a3e640d3a04ea893688f328faedfba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.043160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a3e640d3a04ea893688f328faedfba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.073045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 977/1969] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:27 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:28.077244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.078338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a3e640d3a04ea893688f328faedfba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.079053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.109786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 993/1970] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:27 2022] DELETE /identity/v3/users/16a3e640d3a04ea893688f328faedfba => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:28.155675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d594dc-385c-421e-80b5-c403e58f9d2d admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.167882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:28.252659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.254942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.267359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.268215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.268608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.268990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.270396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.273651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.321931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 978/1971] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/users => generated 288 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:28.326681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.344189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41638364-7c1b-4a90-87d2-5c7775cb70f9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.352721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.354526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.354951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.355354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.356829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.358786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.373533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 979/1972] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:28.378872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.398518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ead8a-c0c1-4913-9d61-7464d7e7a185 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.409129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.410719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.411006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.411419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.412212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.417868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.435718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfd744-b284-4f75-a2d2-32756c835a7e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.448182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.449574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.449895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.450240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] PATH_INFO: `/v3/projects/34200e2b43b2453b81b096689df66555` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.451724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34200e2b43b2453b81b096689df66555)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.455829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 980/1973] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:28.455964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.456879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.469568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca3ed1-f591-4b0c-9e54-f261178e622b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.474577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb622d12-5456-40a0-8cb5-06e16d40d31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.475164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622d12-5456-40a0-8cb5-06e16d40d31d admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.475440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622d12-5456-40a0-8cb5-06e16d40d31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.475742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622d12-5456-40a0-8cb5-06e16d40d31d admin admin] PATH_INFO: `/v3/projects/70e8fdeb7b734729822de6f3012e238c/users/5997660ffd4e4229be6c2c270391bd2a/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.501719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb622d12-5456-40a0-8cb5-06e16d40d31d admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e8fdeb7b734729822de6f3012e238c, user_id=5997660ffd4e4229be6c2c270391bd2a, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.504833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb622d12-5456-40a0-8cb5-06e16d40d31d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.513742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34200e2b43b2453b81b096689df66555', 'request_id': 'req-389e05cf-97ab-409c-b1f0-5c75203775f8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.520682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34200e2b43b2453b81b096689df66555', 'request_id': 'req-389e05cf-97ab-409c-b1f0-5c75203775f8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.530561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 981/1974] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:28 2022] PUT /identity/v3/projects/70e8fdeb7b734729822de6f3012e238c/users/5997660ffd4e4229be6c2c270391bd2a/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:28.530737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34200e2b43b2453b81b096689df66555', 'request_id': 'req-389e05cf-97ab-409c-b1f0-5c75203775f8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.533455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5989bd-9257-4b87-9428-50b10aba47cb None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.533938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5989bd-9257-4b87-9428-50b10aba47cb None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.534585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5989bd-9257-4b87-9428-50b10aba47cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.542774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] The token cache is being invalidate because project 34200e2b43b2453b81b096689df66555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:28.543067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.544254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-389e05cf-97ab-409c-b1f0-5c75203775f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.545690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 994/1975] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:28 2022] DELETE /identity/v3/projects/34200e2b43b2453b81b096689df66555 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:28.554137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.554430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.554703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.571552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9e5989bd-9257-4b87-9428-50b10aba47cb None None] MFA Rules not processed for user `5997660ffd4e4229be6c2c270391bd2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:28.573517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9e5989bd-9257-4b87-9428-50b10aba47cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.578594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:28.579855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.594311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:28.648420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 982/1976] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:56:28.664958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 995/1977] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:28.667270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.668731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5688c2-2f94-4cf0-a86f-db0ba2b2a507 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.669040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5688c2-2f94-4cf0-a86f-db0ba2b2a507 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.669324 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5688c2-2f94-4cf0-a86f-db0ba2b2a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.670367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.691263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-fb5688c2-2f94-4cf0-a86f-db0ba2b2a507 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:28.692362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fb5688c2-2f94-4cf0-a86f-db0ba2b2a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.701176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 983/1978] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:28.705167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5688c2-2f94-4cf0-a86f-db0ba2b2a507 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.715669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5688c2-2f94-4cf0-a86f-db0ba2b2a507 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.721908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.723154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.723154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.723362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.723931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:28.724266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.726792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.739600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 984/1979] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:28.743980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.744101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61a0a-1a4b-4706-8657-6d6994a476a1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.748719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.749424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.749778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.750055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.755681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040ecf8-d684-4f8a-b302-71c3363e6478 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.759842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.760653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.761358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.761471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.761703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.762045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.762734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.764734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.774484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 996/1980] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:56:28.789804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.793284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.814481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472b9c8-4761-4152-b9a4-2472fe2645f8 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.830022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.831065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.831401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.832162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] PATH_INFO: `/v3/projects/1e7e4bb90b8e4fe294918388318daed3` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.834101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7e4bb90b8e4fe294918388318daed3)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.836765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.838168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 985/1981] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:28.842558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.848587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:28.929369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7e4bb90b8e4fe294918388318daed3', 'request_id': 'req-41b96ea3-7148-4cc4-95f2-7120181cf3c5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.938225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7e4bb90b8e4fe294918388318daed3', 'request_id': 'req-41b96ea3-7148-4cc4-95f2-7120181cf3c5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.944058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa9386-6a2a-4177-a1c2-469df246a14a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.957397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7e4bb90b8e4fe294918388318daed3', 'request_id': 'req-41b96ea3-7148-4cc4-95f2-7120181cf3c5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.984973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] The token cache is being invalidate because project 1e7e4bb90b8e4fe294918388318daed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:28.985410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.985963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:28.990913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 997/1982] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:28 2022] DELETE /identity/v3/projects/1e7e4bb90b8e4fe294918388318daed3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:28.993166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.993285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:29.051947 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.052741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.053086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.054129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.054922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.057750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.060023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:29.093376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b96ea3-7148-4cc4-95f2-7120181cf3c5 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:29.099615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.100426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.100782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.101133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] PATH_INFO: `/v3/users/82d0bd4e80094372a8d333a06b8d5c16` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.108195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 986/1983] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:29.108489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82d0bd4e80094372a8d333a06b8d5c16)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.110824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.113528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:29.115615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:29.131978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7424b-ecf9-4d74-ab44-b6344a83701b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:29.141181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.142076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.142192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.142508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.144652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.147396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.165120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 987/1984] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:29.166331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:29.174719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82d0bd4e80094372a8d333a06b8d5c16'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.175463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82d0bd4e80094372a8d333a06b8d5c16'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.179191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611794ac-cba6-42e3-8889-ef17505ec7a8 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:29.184898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.186303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.186521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.186748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.187412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.201385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.201706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82d0bd4e80094372a8d333a06b8d5c16'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.203646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 988/1985] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:29.206350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82d0bd4e80094372a8d333a06b8d5c16'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.208127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:29.210615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:29.222028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c203c82-8b64-4b3c-b018-4ac1dbd42ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82d0bd4e80094372a8d333a06b8d5c16'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.231072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201613a-c543-4443-9008-93fa6a7b6bc9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:29.241343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.241343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.241343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.241343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] PATH_INFO: `/v3/projects/dd5678401aa84013b01e2686614643ef/users/073cab5629fa4d3c8f5bd6bfe0689976/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.264181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 998/1986] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:28 2022] DELETE /identity/v3/users/82d0bd4e80094372a8d333a06b8d5c16 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:29.299531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5678401aa84013b01e2686614643ef, user_id=073cab5629fa4d3c8f5bd6bfe0689976, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.302886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.338639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 989/1987] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:29 2022] PUT /identity/v3/projects/dd5678401aa84013b01e2686614643ef/users/073cab5629fa4d3c8f5bd6bfe0689976/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:29.351683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.351983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.352282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.387190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] MFA Rules not processed for user `073cab5629fa4d3c8f5bd6bfe0689976`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:29.388520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:29.500410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 999/1988] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:29.513704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:29.515598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:29.595160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:29.624044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7d850-8710-40d3-b8fa-2053f8671a0b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:29.628916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15b6f4-3818-49bc-9870-e3981104464b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c76a4a9747469d88a0a1a5748452a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.629507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b6f4-3818-49bc-9870-e3981104464b service glance] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.629948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b6f4-3818-49bc-9870-e3981104464b service glance] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.630229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b6f4-3818-49bc-9870-e3981104464b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.636040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:29.638089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:29.640630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15b6f4-3818-49bc-9870-e3981104464b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.642074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15b6f4-3818-49bc-9870-e3981104464b service glance] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.653477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 990/1989] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:56:29.755399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:29.781557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cf6e4-8de9-4684-adcc-34717f757aaa None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:29.799185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.800020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.800332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.800605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] PATH_INFO: `/v3/projects/2e908bec76494639b66159464c78b4d1` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.801787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e908bec76494639b66159464c78b4d1)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.804181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.881894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e908bec76494639b66159464c78b4d1', 'request_id': 'req-5af793da-ce55-483b-8d76-1e1909cfe3c5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.896068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e908bec76494639b66159464c78b4d1', 'request_id': 'req-5af793da-ce55-483b-8d76-1e1909cfe3c5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.907778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e908bec76494639b66159464c78b4d1', 'request_id': 'req-5af793da-ce55-483b-8d76-1e1909cfe3c5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.931000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] The token cache is being invalidate because project 2e908bec76494639b66159464c78b4d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:29.931302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.932108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-5af793da-ce55-483b-8d76-1e1909cfe3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:29.934395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1000/1990] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:29 2022] DELETE /identity/v3/projects/2e908bec76494639b66159464c78b4d1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:29.939805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23d9c-39bd-48a5-83b6-e63d8287c5c8 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.940081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23d9c-39bd-48a5-83b6-e63d8287c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.940411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23d9c-39bd-48a5-83b6-e63d8287c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.977870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-59c23d9c-39bd-48a5-83b6-e63d8287c5c8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:29.980412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-59c23d9c-39bd-48a5-83b6-e63d8287c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:30.069506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 991/1991] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:30.080088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e854d-d905-45c6-8a02-7dea4084b4ce None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.080408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e854d-d905-45c6-8a02-7dea4084b4ce None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.080839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e854d-d905-45c6-8a02-7dea4084b4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.109641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-595e854d-d905-45c6-8a02-7dea4084b4ce None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:30.110624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-595e854d-d905-45c6-8a02-7dea4084b4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:30.127330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23d9c-39bd-48a5-83b6-e63d8287c5c8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:30.136341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1001/1992] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:30.141691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23d9c-39bd-48a5-83b6-e63d8287c5c8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:30.147179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:30.147818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.150275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.150559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.152359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:30.154144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:30.155945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:30.171428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 992/1993] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:30.176606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e854d-d905-45c6-8a02-7dea4084b4ce None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:30.188647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e854d-d905-45c6-8a02-7dea4084b4ce None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:30.194291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:30.195078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.195557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.195836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.196755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:30.198731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:30.237531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1002/1994] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:30 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:30.239421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:30.251574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a575d-6ccb-410d-9ab2-47d052c6c5d3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:30.428826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:30.429623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.429973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.430323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.431144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:30.433625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:30.489991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 993/1995] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:30 2022] POST /identity/v3/users => generated 283 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:30.494514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:30.516400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919bd8-d0bf-4fda-b80a-0520b459eb35 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:30.522547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:30.523378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.523818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.524269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.525343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:30.527762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:30.544424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1003/1996] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:30.547607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:30.566856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad682698-a505-4c3d-8c5c-f65e3cd1d823 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:30.573387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fad34d-7f46-4abe-9a62-2d6a99b69793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:30.574827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fad34d-7f46-4abe-9a62-2d6a99b69793 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.575215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fad34d-7f46-4abe-9a62-2d6a99b69793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.575675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fad34d-7f46-4abe-9a62-2d6a99b69793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.577160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fad34d-7f46-4abe-9a62-2d6a99b69793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:30.579374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fad34d-7f46-4abe-9a62-2d6a99b69793 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:30.594038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 994/1997] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:30.598655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:30.618886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9b7a-5cae-4ca6-95da-a61a4cec5289 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:30.625260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:30.626278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.626737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.627196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] PATH_INFO: `/v3/projects/ddb8a4af246141779f34d908e3a1de00/users/f6b39e5a8d2042b5b36fde55301dec25/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.649849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb8a4af246141779f34d908e3a1de00, user_id=f6b39e5a8d2042b5b36fde55301dec25, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:30.652853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:30.686150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1004/1998] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:30 2022] PUT /identity/v3/projects/ddb8a4af246141779f34d908e3a1de00/users/f6b39e5a8d2042b5b36fde55301dec25/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:30.697101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e6616-1e89-48e9-8738-0fd6c615ae65 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:30.697394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e6616-1e89-48e9-8738-0fd6c615ae65 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:30.697673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e6616-1e89-48e9-8738-0fd6c615ae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:30.741568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3f2e6616-1e89-48e9-8738-0fd6c615ae65 None None] MFA Rules not processed for user `f6b39e5a8d2042b5b36fde55301dec25`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:30.748096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3f2e6616-1e89-48e9-8738-0fd6c615ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:30.822796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:30.904504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 995/1999] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:30.920387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:30.923401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:31.041470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97941d-bf31-4bd4-9e87-c5c45c312047 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:31.052610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e56a6-3793-4e1b-a0f7-2bf334b999e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:31.053320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e56a6-3793-4e1b-a0f7-2bf334b999e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:31.053600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e56a6-3793-4e1b-a0f7-2bf334b999e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:31.053874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e56a6-3793-4e1b-a0f7-2bf334b999e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:31.065483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e56a6-3793-4e1b-a0f7-2bf334b999e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:31.067529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e56a6-3793-4e1b-a0f7-2bf334b999e1 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:31.086109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1005/2000] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:40.643903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad0642-5640-4481-ad3b-1f2039d82c75 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:40.644481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad0642-5640-4481-ad3b-1f2039d82c75 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:40.644853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad0642-5640-4481-ad3b-1f2039d82c75 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:40.646950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 996/2001] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 16:56:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:40.661870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:40.662381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:40.662475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:40.691013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:40.692431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:40.712682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:40.733105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad0642-5640-4481-ad3b-1f2039d82c75 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:40.737533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-00ad0642-5640-4481-ad3b-1f2039d82c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:40.777244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1006/2002] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 16:56:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:40.788868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:40.790270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:40.805611 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24550db-f4a9-4297-a40e-4b1a424e889c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:40.811938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:40.812752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:40.813033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:40.813302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:40.821987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:40.822881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad0642-5640-4481-ad3b-1f2039d82c75 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:40.823567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:40.827550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:40.828218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:40.828743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:40.829031 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] PATH_INFO: `/v3/users/6677b89e26a94e5da074431e11e12050` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:40.830708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6677b89e26a94e5da074431e11e12050)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:40.833238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:40.835741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1007/2003] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:56:40.885128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6677b89e26a94e5da074431e11e12050'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:40.891984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6677b89e26a94e5da074431e11e12050'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:40.899386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6677b89e26a94e5da074431e11e12050'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:40.900197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6677b89e26a94e5da074431e11e12050'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:40.911856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-920727cf-518c-48ee-bbff-3bcee046f8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6677b89e26a94e5da074431e11e12050'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:40.917571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 997/2004] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:40 2022] DELETE /identity/v3/users/6677b89e26a94e5da074431e11e12050 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:41.101470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.112637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d146f67-c5b6-44a4-869b-d5e855af3fb2 service placement] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.171044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.171606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.171956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.172272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] PATH_INFO: `/v3/projects/25eeef036a7e44b0b5ee4d989f7ec507` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.173528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25eeef036a7e44b0b5ee4d989f7ec507)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.175883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.223118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25eeef036a7e44b0b5ee4d989f7ec507', 'request_id': 'req-786c7068-b1b0-4256-bde1-f4896e499d20'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:41.228972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25eeef036a7e44b0b5ee4d989f7ec507', 'request_id': 'req-786c7068-b1b0-4256-bde1-f4896e499d20'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:41.235213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25eeef036a7e44b0b5ee4d989f7ec507', 'request_id': 'req-786c7068-b1b0-4256-bde1-f4896e499d20'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:41.248582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] The token cache is being invalidate because project 25eeef036a7e44b0b5ee4d989f7ec507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:41.248877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:41.249502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-786c7068-b1b0-4256-bde1-f4896e499d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:41.251000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1008/2005] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:41 2022] DELETE /identity/v3/projects/25eeef036a7e44b0b5ee4d989f7ec507 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:41.255951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6076dd-4281-44f4-8ae9-c802797e4d42 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.256267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6076dd-4281-44f4-8ae9-c802797e4d42 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.256546 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6076dd-4281-44f4-8ae9-c802797e4d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.276813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ab6076dd-4281-44f4-8ae9-c802797e4d42 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:41.278012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ab6076dd-4281-44f4-8ae9-c802797e4d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:41.325869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:41.342846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 998/2006] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:41.350576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c41f9-3996-41c0-8f1e-cfafd810bdf6 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.350869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c41f9-3996-41c0-8f1e-cfafd810bdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.351178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c41f9-3996-41c0-8f1e-cfafd810bdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.372579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-581c41f9-3996-41c0-8f1e-cfafd810bdf6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:41.373930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-581c41f9-3996-41c0-8f1e-cfafd810bdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:41.381876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1009/2007] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:41.385987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6076dd-4281-44f4-8ae9-c802797e4d42 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.395586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6076dd-4281-44f4-8ae9-c802797e4d42 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.400222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.400814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.401095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.401372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.402016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:41.402407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.404528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.429240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 999/2008] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:41.431424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c41f9-3996-41c0-8f1e-cfafd810bdf6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.441592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c41f9-3996-41c0-8f1e-cfafd810bdf6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.446395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.446997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.447279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.447553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.448215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.450133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.478660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1010/2009] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:41 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:41.483761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.494679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc502-0a68-4384-a6ce-eb368c162fc7 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.551032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.551647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.551991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.552254 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.552872 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.554775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.584436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1000/2010] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:41 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:41.588535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.599578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90dd41-c110-4790-8b98-3f1a67b63789 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.604303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.604910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.605215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.606005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.606642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.608247 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.609528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:41.623402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1011/2011] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:41.624727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.634690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55adad-a785-4870-8f15-529aa8bb6a8a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.639544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a9342-89d3-42bd-8881-17cac63ceff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.640165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a9342-89d3-42bd-8881-17cac63ceff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.640466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a9342-89d3-42bd-8881-17cac63ceff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.640766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a9342-89d3-42bd-8881-17cac63ceff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.641484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a9342-89d3-42bd-8881-17cac63ceff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.643108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a9342-89d3-42bd-8881-17cac63ceff9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.654984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1001/2012] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:41.659078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.670181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13806dff-d8a4-41b0-b84e-4770bc24b756 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.675219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.675924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.676228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.676505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] PATH_INFO: `/v3/projects/1397153721b448b29c0e9ef89ea95003/users/44ef2277d9ae434d94edc138ef033916/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.692444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1397153721b448b29c0e9ef89ea95003, user_id=44ef2277d9ae434d94edc138ef033916, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.695097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.709430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1012/2013] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:41 2022] PUT /identity/v3/projects/1397153721b448b29c0e9ef89ea95003/users/44ef2277d9ae434d94edc138ef033916/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:41.720012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a3575-45a6-4cad-bb05-f079b736801f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.720337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a3575-45a6-4cad-bb05-f079b736801f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.720616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a3575-45a6-4cad-bb05-f079b736801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.747904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-169a3575-45a6-4cad-bb05-f079b736801f None None] MFA Rules not processed for user `44ef2277d9ae434d94edc138ef033916`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:41.749162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-169a3575-45a6-4cad-bb05-f079b736801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:41.813102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1002/2014] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:41.817680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.882777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198cecb-7182-4728-a01a-e40571a33392 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.887494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.888104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.888416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.888722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.889356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.891261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.919718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1013/2015] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:41 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:41.923708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a3575-45a6-4cad-bb05-f079b736801f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.934414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a3575-45a6-4cad-bb05-f079b736801f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.982461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:41.997345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.997956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.998236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.998502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.999116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:42.001086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:42.035480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1003/2016] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:41 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:42.039371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:42.050841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d7dc-8efe-4030-9d9c-ed6453be97ac admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:42.055497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:42.056100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:42.056410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:42.056681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:42.057351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:42.059500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:42.070151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1014/2017] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:42.074978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:42.085361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e0c55-9898-435f-b1b9-c3842158bfe2 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:42.090444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:42.091253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:42.091538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:42.091811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] PATH_INFO: `/v3/projects/32bc5b6af7d646bba9456ceca5a3fe47/users/407216ac389542b380b48fac5f537d4f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:42.108276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bc5b6af7d646bba9456ceca5a3fe47, user_id=407216ac389542b380b48fac5f537d4f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:42.111074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:42.125290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1004/2018] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:42 2022] PUT /identity/v3/projects/32bc5b6af7d646bba9456ceca5a3fe47/users/407216ac389542b380b48fac5f537d4f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:42.129415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:42.139367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a4f51-c7f3-4029-a50a-3e57d947969a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:42.195037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db98fe-40e7-45af-aab2-d9069af05ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:42.195666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db98fe-40e7-45af-aab2-d9069af05ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:42.195945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db98fe-40e7-45af-aab2-d9069af05ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:42.196292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db98fe-40e7-45af-aab2-d9069af05ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:42.196912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db98fe-40e7-45af-aab2-d9069af05ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:42.198532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db98fe-40e7-45af-aab2-d9069af05ff2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:42.209439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1015/2019] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:42.213606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:42.223975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84973f20-4c37-4af2-a061-f28450e30e9e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:42.228590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:42.229237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:42.229516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:42.229789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] PATH_INFO: `/v3/projects/32bc5b6af7d646bba9456ceca5a3fe47/users/407216ac389542b380b48fac5f537d4f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:42.232424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bc5b6af7d646bba9456ceca5a3fe47, user_id=407216ac389542b380b48fac5f537d4f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:42.235218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:42.245183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1005/2020] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:42 2022] PUT /identity/v3/projects/32bc5b6af7d646bba9456ceca5a3fe47/users/407216ac389542b380b48fac5f537d4f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:42.258253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecddc22-4c9e-4afd-86b5-3e2ca8a33e19 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:42.258552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecddc22-4c9e-4afd-86b5-3e2ca8a33e19 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:42.258914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecddc22-4c9e-4afd-86b5-3e2ca8a33e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:42.285889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0ecddc22-4c9e-4afd-86b5-3e2ca8a33e19 None None] MFA Rules not processed for user `407216ac389542b380b48fac5f537d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:42.287104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0ecddc22-4c9e-4afd-86b5-3e2ca8a33e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:42.317518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:42.361304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1016/2021] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:42.373110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:42.374803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:42.432058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc9a1-789c-46a1-944f-577335b77411 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:42.436783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:42.437430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:42.437711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:42.437983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:42.447740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:42.449259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:42.458964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1006/2022] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:42.532215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecddc22-4c9e-4afd-86b5-3e2ca8a33e19 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:42.541699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecddc22-4c9e-4afd-86b5-3e2ca8a33e19 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:42.546057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255f1d6-8e30-47f4-802c-4c6efe72d442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:42.546664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f1d6-8e30-47f4-802c-4c6efe72d442 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:42.546949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f1d6-8e30-47f4-802c-4c6efe72d442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:42.547227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f1d6-8e30-47f4-802c-4c6efe72d442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:42.604710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255f1d6-8e30-47f4-802c-4c6efe72d442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:42.606733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255f1d6-8e30-47f4-802c-4c6efe72d442 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:42.615795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1017/2023] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:48.025617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:48.027056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:48.092590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d131-d129-4842-b7d0-17486956a016 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:48.097200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:48.097779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.098063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.098340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] PATH_INFO: `/v3/users/5997660ffd4e4229be6c2c270391bd2a` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.099507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5997660ffd4e4229be6c2c270391bd2a)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:48.101482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:48.147466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:48.150112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5997660ffd4e4229be6c2c270391bd2a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.159068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5997660ffd4e4229be6c2c270391bd2a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.168559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5997660ffd4e4229be6c2c270391bd2a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.169256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5997660ffd4e4229be6c2c270391bd2a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.181617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5997660ffd4e4229be6c2c270391bd2a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.188539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1007/2024] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:56:48 2022] DELETE /identity/v3/users/5997660ffd4e4229be6c2c270391bd2a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:48.193016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.193512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.193602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.212329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:48.213510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:48.274088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1018/2025] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:48.283615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:48.285026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:48.345955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4572-f1b1-4715-9b3b-b5481ffe9d28 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:48.349704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178d69c-b1b6-45a7-98ad-96e4e36c61e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:48.350328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178d69c-b1b6-45a7-98ad-96e4e36c61e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.350641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178d69c-b1b6-45a7-98ad-96e4e36c61e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.350919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178d69c-b1b6-45a7-98ad-96e4e36c61e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.361233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178d69c-b1b6-45a7-98ad-96e4e36c61e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:48.362661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178d69c-b1b6-45a7-98ad-96e4e36c61e2 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:48.372993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1008/2026] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:48.696875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:48.698240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:48.712137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b80cca-fa98-4f78-8ead-de746f5ba504 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:48.736906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:48.737573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.737866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.738139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] PATH_INFO: `/v3/projects/70e8fdeb7b734729822de6f3012e238c` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.739606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e8fdeb7b734729822de6f3012e238c)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:48.741627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:48.798438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e8fdeb7b734729822de6f3012e238c', 'request_id': 'req-8f5829d0-6ef4-4f3a-9956-3b38525c4000'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.804577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e8fdeb7b734729822de6f3012e238c', 'request_id': 'req-8f5829d0-6ef4-4f3a-9956-3b38525c4000'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.807203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:48.813135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e8fdeb7b734729822de6f3012e238c', 'request_id': 'req-8f5829d0-6ef4-4f3a-9956-3b38525c4000'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.826734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] The token cache is being invalidate because project 70e8fdeb7b734729822de6f3012e238c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:56:48.827029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.827762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8f5829d0-6ef4-4f3a-9956-3b38525c4000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:56:48.829456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1019/2027] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:56:48 2022] DELETE /identity/v3/projects/70e8fdeb7b734729822de6f3012e238c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:48.835118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e576658-8e15-43c0-b428-8956baf00761 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.835406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e576658-8e15-43c0-b428-8956baf00761 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.835680 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e576658-8e15-43c0-b428-8956baf00761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.855870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1e576658-8e15-43c0-b428-8956baf00761 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:48.857113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1e576658-8e15-43c0-b428-8956baf00761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:48.928071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1009/2028] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:56:48.936373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053a6fc-2795-4ce7-b8c5-7e5a2a6ec7f0 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.936763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053a6fc-2795-4ce7-b8c5-7e5a2a6ec7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.937169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053a6fc-2795-4ce7-b8c5-7e5a2a6ec7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.959525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b053a6fc-2795-4ce7-b8c5-7e5a2a6ec7f0 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:48.960897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b053a6fc-2795-4ce7-b8c5-7e5a2a6ec7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:48.981290 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1020/2029] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:48.985388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e576658-8e15-43c0-b428-8956baf00761 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:48.995903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e576658-8e15-43c0-b428-8956baf00761 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:49.016248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:49.017086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.017452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.017802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.018535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:49.018908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:49.020646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:49.038774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053a6fc-2795-4ce7-b8c5-7e5a2a6ec7f0 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:49.040658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1010/2030] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:49.051387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053a6fc-2795-4ce7-b8c5-7e5a2a6ec7f0 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:49.058068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:49.059123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.059499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.059870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.060615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:49.063096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:49.098411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1021/2031] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:56:49 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:49.099990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:49.111844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46b30-bc57-4df6-89ab-e4c973936c9d admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:49.160574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:49.181262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:49.181262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.181262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.181262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.181578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:49.181578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:49.206758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1011/2032] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:49 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:49.211208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:49.222161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c43f0b-03fa-46c4-9b6f-487fcc4edee0 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:49.231244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:49.231244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.231244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.231244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.231515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:49.231515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:49.243983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1022/2033] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:49.245197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:49.256127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c971c3-6cb8-4435-953e-a57d072f4d19 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:49.261064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d928-e0a9-4297-93fa-d6c8cc9f39c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:49.262248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d928-e0a9-4297-93fa-d6c8cc9f39c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.262526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d928-e0a9-4297-93fa-d6c8cc9f39c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.262799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d928-e0a9-4297-93fa-d6c8cc9f39c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.263478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9d928-e0a9-4297-93fa-d6c8cc9f39c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:49.265126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9d928-e0a9-4297-93fa-d6c8cc9f39c2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:49.275817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1012/2034] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:56:49.279979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:49.290189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394874-13a6-474b-8325-2eb4448d632a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:49.295093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:49.295744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.296021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.296332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] PATH_INFO: `/v3/projects/aa1a38abca5848b78518b882982fc1be/users/409b789be2d9402885b2498435c76286/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.311364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1a38abca5848b78518b882982fc1be, user_id=409b789be2d9402885b2498435c76286, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:49.314148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:49.328129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1023/2035] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:56:49 2022] PUT /identity/v3/projects/aa1a38abca5848b78518b882982fc1be/users/409b789be2d9402885b2498435c76286/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:49.333498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d15e-bd47-46a8-8e25-bdc3a7714b67 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.333789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d15e-bd47-46a8-8e25-bdc3a7714b67 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.334092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d15e-bd47-46a8-8e25-bdc3a7714b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.353221 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6e73d15e-bd47-46a8-8e25-bdc3a7714b67 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:49.354443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6e73d15e-bd47-46a8-8e25-bdc3a7714b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:49.415865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1013/2036] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:49.426132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:49.428115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:49.496316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1e7fd-2a53-4070-9931-e5420f4ce06f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:49.500213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:49.501126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:49.501303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:49.501426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:49.511351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:49.513390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:49.523799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1024/2037] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:56:50.234890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b11a9-4789-418c-b665-725c7786b168 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:50.235349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b11a9-4789-418c-b665-725c7786b168 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:50.235743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b11a9-4789-418c-b665-725c7786b168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:50.267968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9d6b11a9-4789-418c-b665-725c7786b168 None None] MFA Rules not processed for user `409b789be2d9402885b2498435c76286`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:50.269287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9d6b11a9-4789-418c-b665-725c7786b168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:50.342481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1014/2038] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:56:50.353426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:50.354806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:50.415393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40522f-4094-4a08-abfc-1874b6ae7deb service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:50.419131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:50.419740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:50.420022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:50.420562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:50.429601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:50.431025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:50.440129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1025/2039] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:56:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:56:57.116627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4446a-679e-4903-8075-a4323f21976d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:57.116627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4446a-679e-4903-8075-a4323f21976d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:57.116627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4446a-679e-4903-8075-a4323f21976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:57.137561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-fbe4446a-679e-4903-8075-a4323f21976d None None] MFA Rules not processed for user `409b789be2d9402885b2498435c76286`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:57.138433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fbe4446a-679e-4903-8075-a4323f21976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:57.146080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1015/2040] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:56:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:00.435925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:00.491945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:00.493452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:00.511914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3c51d-1337-4ffa-bdbb-848a89e10ba4 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:00.516863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:00.517645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:00.517922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:00.518373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] PATH_INFO: `/v3/users/44ef2277d9ae434d94edc138ef033916` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:00.519752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=44ef2277d9ae434d94edc138ef033916)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:00.522292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:00.574305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44ef2277d9ae434d94edc138ef033916'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:00.574979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44ef2277d9ae434d94edc138ef033916'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:00.583853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44ef2277d9ae434d94edc138ef033916'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:00.584591 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44ef2277d9ae434d94edc138ef033916'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:00.599708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44ef2277d9ae434d94edc138ef033916'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:00.613534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1026/2041] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:00 2022] DELETE /identity/v3/users/44ef2277d9ae434d94edc138ef033916 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:00.618208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:00.619274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:00.619832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:00.640247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:00.641536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:00.698098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:00.714425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1016/2042] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:00.724734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:00.726435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:00.801467 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a1a6e-5895-4cbe-a6d8-39871eb262cb admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:00.816803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:00.817446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:00.817724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:00.817995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:00.828496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:00.829964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:00.843156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1027/2043] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:01.339381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:01.341229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:01.355754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c0cb3-150b-42cc-b94f-a2f3aa66cf37 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:01.361247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:01.362032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:01.362473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:01.362815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] PATH_INFO: `/v3/projects/1397153721b448b29c0e9ef89ea95003` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:01.364228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1397153721b448b29c0e9ef89ea95003)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:01.366358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:01.418521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1397153721b448b29c0e9ef89ea95003', 'request_id': 'req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.424363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1397153721b448b29c0e9ef89ea95003', 'request_id': 'req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.438501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1397153721b448b29c0e9ef89ea95003', 'request_id': 'req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.454162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] The token cache is being invalidate because project 1397153721b448b29c0e9ef89ea95003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:01.454488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.455168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.456855 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1017/2044] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:01 2022] DELETE /identity/v3/projects/1397153721b448b29c0e9ef89ea95003 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:01.461257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:01.471443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5d25c-cd2f-43ca-8505-71d73ed70884 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:01.530418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:01.531055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:01.531338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:01.531610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] PATH_INFO: `/v3/users/407216ac389542b380b48fac5f537d4f` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:01.532762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=407216ac389542b380b48fac5f537d4f)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:01.534689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:01.554914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:01.588384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '407216ac389542b380b48fac5f537d4f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.589030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '407216ac389542b380b48fac5f537d4f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.596502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '407216ac389542b380b48fac5f537d4f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.597217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '407216ac389542b380b48fac5f537d4f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.610670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-68eeedcd-2440-4039-8133-96d269edeffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '407216ac389542b380b48fac5f537d4f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:01.625772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1028/2045] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:01 2022] DELETE /identity/v3/users/407216ac389542b380b48fac5f537d4f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:01.916102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:01.918144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:01.931727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4d6fa-17c5-4309-aeaf-f3d17fd270f2 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:01.994006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:01.994794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:01.995252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:01.995649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] PATH_INFO: `/v3/projects/32bc5b6af7d646bba9456ceca5a3fe47` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:01.996943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bc5b6af7d646bba9456ceca5a3fe47)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:01.999046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.066784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bc5b6af7d646bba9456ceca5a3fe47', 'request_id': 'req-3b32a402-478a-4673-8d79-60d1be6f7151'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:02.073525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bc5b6af7d646bba9456ceca5a3fe47', 'request_id': 'req-3b32a402-478a-4673-8d79-60d1be6f7151'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:02.081004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bc5b6af7d646bba9456ceca5a3fe47', 'request_id': 'req-3b32a402-478a-4673-8d79-60d1be6f7151'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:02.099791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] The token cache is being invalidate because project 32bc5b6af7d646bba9456ceca5a3fe47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:02.100091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:02.100792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3b32a402-478a-4673-8d79-60d1be6f7151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:02.102348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1018/2046] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:01 2022] DELETE /identity/v3/projects/32bc5b6af7d646bba9456ceca5a3fe47 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:02.108246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48fe59-efdd-4cf8-b57a-8db3f6aac9cf None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.108617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48fe59-efdd-4cf8-b57a-8db3f6aac9cf None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.108986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48fe59-efdd-4cf8-b57a-8db3f6aac9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.129028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-fd48fe59-efdd-4cf8-b57a-8db3f6aac9cf None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:02.130397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fd48fe59-efdd-4cf8-b57a-8db3f6aac9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:02.189507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1029/2047] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:02.197582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61b90-5d14-4cd3-956b-0658eb4ccbca None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.197890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61b90-5d14-4cd3-956b-0658eb4ccbca None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.198198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61b90-5d14-4cd3-956b-0658eb4ccbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.213622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:02.219815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-14d61b90-5d14-4cd3-956b-0658eb4ccbca None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:02.221606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-14d61b90-5d14-4cd3-956b-0658eb4ccbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:02.229991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1019/2048] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:02.235434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48fe59-efdd-4cf8-b57a-8db3f6aac9cf None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.245898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48fe59-efdd-4cf8-b57a-8db3f6aac9cf None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.251873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.252420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.252703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.252979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.253635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:02.253943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.255537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.267826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1030/2049] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:57:02.272227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61b90-5d14-4cd3-956b-0658eb4ccbca None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.282341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61b90-5d14-4cd3-956b-0658eb4ccbca None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.287023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.287607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.287882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.288299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.288939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.290933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.319680 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1020/2050] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:02 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:02.323603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.334555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c34e44-4009-4b65-9a99-f2ff8e97678e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.395414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.395534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.395759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.396100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.396691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.398810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.429615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1031/2051] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:02 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:02.433865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.448843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1a37a-6398-485c-ae9d-5875ce69e301 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.454417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.455047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.456498 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.456856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.457599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.459563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.472367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1021/2052] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:02.476791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.487098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7f9e0-c0b7-43ee-9937-4b2317630443 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.492035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700249c-ea79-4bd2-88ab-aabfc3d54c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.492733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700249c-ea79-4bd2-88ab-aabfc3d54c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.493088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700249c-ea79-4bd2-88ab-aabfc3d54c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.493431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700249c-ea79-4bd2-88ab-aabfc3d54c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.494164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700249c-ea79-4bd2-88ab-aabfc3d54c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.495816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700249c-ea79-4bd2-88ab-aabfc3d54c73 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.501480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:02.507276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1032/2053] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:02.511261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.522232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553312-9a6a-4a62-91d1-fb2a9d535585 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.526833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.527530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.527812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.528097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] PATH_INFO: `/v3/projects/01ccd61fc0b549f98e44be273d5da5b5/users/793b8355ffdb46c382b6b2207ff740e8/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.543930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ccd61fc0b549f98e44be273d5da5b5, user_id=793b8355ffdb46c382b6b2207ff740e8, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.546620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.558973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1022/2054] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:02 2022] PUT /identity/v3/projects/01ccd61fc0b549f98e44be273d5da5b5/users/793b8355ffdb46c382b6b2207ff740e8/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:02.570444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0a35-a910-4e96-b00e-8f640966f45e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.570729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0a35-a910-4e96-b00e-8f640966f45e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.571002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0a35-a910-4e96-b00e-8f640966f45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.606138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-425c0a35-a910-4e96-b00e-8f640966f45e None None] MFA Rules not processed for user `793b8355ffdb46c382b6b2207ff740e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:02.608384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-425c0a35-a910-4e96-b00e-8f640966f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:02.676807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1033/2055] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:02.681356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.742677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7152b-0eae-4692-b8e9-5c029b49f8d5 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.747846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.748482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.748762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.749039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.749658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.751550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.777529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1023/2056] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:02 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:02.781602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c0a35-a910-4e96-b00e-8f640966f45e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.792080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c0a35-a910-4e96-b00e-8f640966f45e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.850724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.851335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.851611 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.851931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.852594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.854479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.876590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1034/2057] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:02 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:02.880602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.895888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713768-e64d-4e7b-9f64-d400a98784de admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.901175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.901933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.902215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.902480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.903693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.905306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.916420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1024/2058] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:02.922075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.933712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f04dd-fae4-422b-a3ad-61bc17d49b8d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:02.938430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:02.939091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:02.939370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:02.939642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] PATH_INFO: `/v3/projects/7a868f8ebf1f4e80b1999f8e80475207/users/59e9ff0d5edc4b96ba0b3d804eff9d4e/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:02.954702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a868f8ebf1f4e80b1999f8e80475207, user_id=59e9ff0d5edc4b96ba0b3d804eff9d4e, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:02.957407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:02.972729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1035/2059] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:02 2022] PUT /identity/v3/projects/7a868f8ebf1f4e80b1999f8e80475207/users/59e9ff0d5edc4b96ba0b3d804eff9d4e/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:57:02.976839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:02.987775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0ef50-c869-46f7-9cd6-6c66f0d74b41 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:03.045696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bc31c-6ec0-4bf0-b58c-ec68410402dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:03.046362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bc31c-6ec0-4bf0-b58c-ec68410402dd admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:03.046646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bc31c-6ec0-4bf0-b58c-ec68410402dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:03.046917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bc31c-6ec0-4bf0-b58c-ec68410402dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:03.047664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bc31c-6ec0-4bf0-b58c-ec68410402dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:03.049416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bc31c-6ec0-4bf0-b58c-ec68410402dd admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:03.059627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1025/2060] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:03.064131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:03.074696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cd3bc-0107-4759-85f4-80569981bb96 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:03.079804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:03.080485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:03.080764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:03.081033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] PATH_INFO: `/v3/projects/7a868f8ebf1f4e80b1999f8e80475207/users/59e9ff0d5edc4b96ba0b3d804eff9d4e/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:03.083174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a868f8ebf1f4e80b1999f8e80475207, user_id=59e9ff0d5edc4b96ba0b3d804eff9d4e, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:03.085827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:03.096792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1036/2061] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:03 2022] PUT /identity/v3/projects/7a868f8ebf1f4e80b1999f8e80475207/users/59e9ff0d5edc4b96ba0b3d804eff9d4e/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:03.107217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769cf3-faae-49ac-8580-5fac10855a3f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:03.107505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769cf3-faae-49ac-8580-5fac10855a3f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:03.107786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769cf3-faae-49ac-8580-5fac10855a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:03.132913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:03.136082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-88769cf3-faae-49ac-8580-5fac10855a3f None None] MFA Rules not processed for user `59e9ff0d5edc4b96ba0b3d804eff9d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:03.138817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-88769cf3-faae-49ac-8580-5fac10855a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:03.210398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1026/2062] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:03.222223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:03.223941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:03.271155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:03.285897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169b2a-bd79-4cb4-b656-46a78ef025b1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:03.290217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:03.290835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:03.291114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:03.291386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:03.301119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:03.302557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:03.313019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1037/2063] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:03.419543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769cf3-faae-49ac-8580-5fac10855a3f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:03.429435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769cf3-faae-49ac-8580-5fac10855a3f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:03.433094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6c60f-c119-487a-8017-a6dc0a73d39a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:03.433702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6c60f-c119-487a-8017-a6dc0a73d39a service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:03.433982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6c60f-c119-487a-8017-a6dc0a73d39a service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:03.434257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6c60f-c119-487a-8017-a6dc0a73d39a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:03.495664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6c60f-c119-487a-8017-a6dc0a73d39a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:03.497650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6c60f-c119-487a-8017-a6dc0a73d39a service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:03.506985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1027/2064] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:07.182725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:07.248034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14bf98-6faf-4e5e-ace0-71d3e4500d14 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:07.252753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:07.253394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:07.253672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:07.253944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] PATH_INFO: `/v3/users/793b8355ffdb46c382b6b2207ff740e8` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:07.255059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793b8355ffdb46c382b6b2207ff740e8)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:07.257035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:07.304868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793b8355ffdb46c382b6b2207ff740e8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.305586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793b8355ffdb46c382b6b2207ff740e8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.313589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793b8355ffdb46c382b6b2207ff740e8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.314226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793b8355ffdb46c382b6b2207ff740e8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.328188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793b8355ffdb46c382b6b2207ff740e8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.341241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1038/2065] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:07 2022] DELETE /identity/v3/users/793b8355ffdb46c382b6b2207ff740e8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:07.345528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:07.346034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:07.346127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:07.367108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:07.368579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:07.442673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1028/2066] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:07.452425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:07.453860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:07.512173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aff01a-b2fd-4498-b7cf-4b7447ab4151 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:07.516346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:07.516965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:07.517247 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:07.517524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:07.526427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:07.527868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:07.538291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1039/2067] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:07.810262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:07.811918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:07.834376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e465a44-b314-4e62-83a0-f19abffc9dd6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:07.839265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:07.839862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:07.840159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:07.840474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] PATH_INFO: `/v3/projects/01ccd61fc0b549f98e44be273d5da5b5` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:07.842270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ccd61fc0b549f98e44be273d5da5b5)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:07.844262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:07.881345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:07.898662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ccd61fc0b549f98e44be273d5da5b5', 'request_id': 'req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.904911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ccd61fc0b549f98e44be273d5da5b5', 'request_id': 'req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.911460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ccd61fc0b549f98e44be273d5da5b5', 'request_id': 'req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.924514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] The token cache is being invalidate because project 01ccd61fc0b549f98e44be273d5da5b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:07.924822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.925441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:07.927062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1029/2068] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:07 2022] DELETE /identity/v3/projects/01ccd61fc0b549f98e44be273d5da5b5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:07.931430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:07.942261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d6a3-caef-413e-b789-6a9eb8bd1803 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:07.946272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:08.011377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:08.012137 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.013146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.015689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] PATH_INFO: `/v3/users/59e9ff0d5edc4b96ba0b3d804eff9d4e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.015689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59e9ff0d5edc4b96ba0b3d804eff9d4e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:08.017538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:08.067142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59e9ff0d5edc4b96ba0b3d804eff9d4e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.067718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59e9ff0d5edc4b96ba0b3d804eff9d4e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.075701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59e9ff0d5edc4b96ba0b3d804eff9d4e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.076535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59e9ff0d5edc4b96ba0b3d804eff9d4e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.090952 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8667ebf7-70b2-47d6-b1f5-6c7fc930ae8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59e9ff0d5edc4b96ba0b3d804eff9d4e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.104729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1040/2069] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:07 2022] DELETE /identity/v3/users/59e9ff0d5edc4b96ba0b3d804eff9d4e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:08.471352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:08.477482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:08.491668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ede7dc-6ec8-4e60-a374-15bbb90a91d9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:08.554555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:08.555200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.555482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.555765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] PATH_INFO: `/v3/projects/7a868f8ebf1f4e80b1999f8e80475207` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.556923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a868f8ebf1f4e80b1999f8e80475207)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:08.558873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:08.606689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a868f8ebf1f4e80b1999f8e80475207', 'request_id': 'req-27312aa5-434c-4155-b119-169b3683e8e4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.612876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a868f8ebf1f4e80b1999f8e80475207', 'request_id': 'req-27312aa5-434c-4155-b119-169b3683e8e4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.619297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a868f8ebf1f4e80b1999f8e80475207', 'request_id': 'req-27312aa5-434c-4155-b119-169b3683e8e4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.633360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] The token cache is being invalidate because project 7a868f8ebf1f4e80b1999f8e80475207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:08.633653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.634319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-27312aa5-434c-4155-b119-169b3683e8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:08.636095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1030/2070] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:08 2022] DELETE /identity/v3/projects/7a868f8ebf1f4e80b1999f8e80475207 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:08.641493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a5603-1aff-43ec-bfcc-fe72f91eb804 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.641981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a5603-1aff-43ec-bfcc-fe72f91eb804 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.642219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a5603-1aff-43ec-bfcc-fe72f91eb804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.663088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-bc6a5603-1aff-43ec-bfcc-fe72f91eb804 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:08.664390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bc6a5603-1aff-43ec-bfcc-fe72f91eb804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:08.723725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1041/2071] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:08.731999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1612631-adfe-466c-84af-46460c3e04a6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.732347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1612631-adfe-466c-84af-46460c3e04a6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.732625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1612631-adfe-466c-84af-46460c3e04a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.754795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d1612631-adfe-466c-84af-46460c3e04a6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:08.756035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d1612631-adfe-466c-84af-46460c3e04a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:08.768537 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1031/2072] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:08.774118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a5603-1aff-43ec-bfcc-fe72f91eb804 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:08.787485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a5603-1aff-43ec-bfcc-fe72f91eb804 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:08.792190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:08.792807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.793090 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.793366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.794679 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:08.794984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:08.796629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:08.808443 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1042/2073] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:57:08.813461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1612631-adfe-466c-84af-46460c3e04a6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:08.823146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1612631-adfe-466c-84af-46460c3e04a6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:08.828760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:08.829361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.829636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.829909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.830542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:08.832466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:08.860999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1032/2074] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:08 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:08.865099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:08.875433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5804b-89c1-4c5c-ba6f-7acdd8dc9a94 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:08.884257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:08.936956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:08.937577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.937911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.938210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.938872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:08.940831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:08.962657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1043/2075] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:08 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:08.967490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:08.978109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dd50d-3d9d-4837-b9c7-91feaaac781e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:08.983468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:08.984075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:08.984382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:08.984657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:08.985306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:08.986935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:08.997260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1033/2076] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:09.001444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.011933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71c29f-a588-4973-bc0b-38a1600fa9c4 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.017804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a25d-37a3-4fc4-a025-0e7c4092bd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.018426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a25d-37a3-4fc4-a025-0e7c4092bd06 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.018709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a25d-37a3-4fc4-a025-0e7c4092bd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.018980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a25d-37a3-4fc4-a025-0e7c4092bd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.019629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825a25d-37a3-4fc4-a025-0e7c4092bd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.021276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825a25d-37a3-4fc4-a025-0e7c4092bd06 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.032903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1044/2077] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:09.037012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.047917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a0b3a-0cbe-4630-aef2-dc44976059dc admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.053042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.053711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.053998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.054260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] PATH_INFO: `/v3/projects/389c509d7a7845f1a03bf6e0ae27e6cf/users/6973b0b65c9b46fa8850e9920c9527ba/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.063009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:09.073080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=389c509d7a7845f1a03bf6e0ae27e6cf, user_id=6973b0b65c9b46fa8850e9920c9527ba, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.077205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.105659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1034/2078] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:09 2022] PUT /identity/v3/projects/389c509d7a7845f1a03bf6e0ae27e6cf/users/6973b0b65c9b46fa8850e9920c9527ba/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:09.118947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c49b89-7b8b-4254-826a-786d33118c76 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.120571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c49b89-7b8b-4254-826a-786d33118c76 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.120571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c49b89-7b8b-4254-826a-786d33118c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.156325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b2c49b89-7b8b-4254-826a-786d33118c76 None None] MFA Rules not processed for user `6973b0b65c9b46fa8850e9920c9527ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:09.157649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b2c49b89-7b8b-4254-826a-786d33118c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:09.256460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1045/2079] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:09.260839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.336478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09725c6-64b0-443f-9bd6-1e573fba7fd6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.342033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.342712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.342995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.343886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.344714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.346630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.374108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1035/2080] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:09 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:09.378191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c49b89-7b8b-4254-826a-786d33118c76 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.389790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c49b89-7b8b-4254-826a-786d33118c76 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.450981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.451616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.451900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.452207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.452844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.454775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.479117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1046/2081] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:09 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:09.485487 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.499000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075bfe4-0777-4641-afab-81abd4256587 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.503664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.504304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.504582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.504875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.505510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.507136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.519401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1036/2082] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:09.523640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.533933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ed483-9a92-49ce-8cb9-1b35de1efa9c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.538627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.539286 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.539564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.539857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] PATH_INFO: `/v3/projects/4e2be38532cc4090bf0ffec16c6c5332/users/21a9771ef36d4168a059ac1508c6ed73/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.549120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:09.557409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2be38532cc4090bf0ffec16c6c5332, user_id=21a9771ef36d4168a059ac1508c6ed73, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.560612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.574570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1047/2083] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:09 2022] PUT /identity/v3/projects/4e2be38532cc4090bf0ffec16c6c5332/users/21a9771ef36d4168a059ac1508c6ed73/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:57:09.579217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.591972 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2249d9-4788-4898-bc4a-0d401cf4af46 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.657628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11716-d8c2-41b5-937d-7b90323e1b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.658241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11716-d8c2-41b5-937d-7b90323e1b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.658519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11716-d8c2-41b5-937d-7b90323e1b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.658789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11716-d8c2-41b5-937d-7b90323e1b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.659428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11716-d8c2-41b5-937d-7b90323e1b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.661052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11716-d8c2-41b5-937d-7b90323e1b35 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.671269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1037/2084] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:57:09.675485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.686278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060c7c6-d128-4d5c-85c5-3be4f98732cf admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.690904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.691545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.691820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.692088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] PATH_INFO: `/v3/projects/4e2be38532cc4090bf0ffec16c6c5332/users/21a9771ef36d4168a059ac1508c6ed73/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.693979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2be38532cc4090bf0ffec16c6c5332, user_id=21a9771ef36d4168a059ac1508c6ed73, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.696637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.707314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1048/2085] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:09 2022] PUT /identity/v3/projects/4e2be38532cc4090bf0ffec16c6c5332/users/21a9771ef36d4168a059ac1508c6ed73/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:09.717777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebcef5-5448-42b9-aab9-9bfac690ce45 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.718063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebcef5-5448-42b9-aab9-9bfac690ce45 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.718337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebcef5-5448-42b9-aab9-9bfac690ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.749236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8aebcef5-5448-42b9-aab9-9bfac690ce45 None None] MFA Rules not processed for user `21a9771ef36d4168a059ac1508c6ed73`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:09.750478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8aebcef5-5448-42b9-aab9-9bfac690ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:09.800063 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:09.828045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1038/2086] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:09.841173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:09.842862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:09.901190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c64486-f444-4c55-9456-ed4858b10c2a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:09.904882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:09.905499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:09.905781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:09.906053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:09.915265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:09.916732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:09.928454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1049/2087] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:10.012564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebcef5-5448-42b9-aab9-9bfac690ce45 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:10.022067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebcef5-5448-42b9-aab9-9bfac690ce45 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:10.025809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5670a-1114-4ab4-889b-0270257bb28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:10.026406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5670a-1114-4ab4-889b-0270257bb28c service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:10.026714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5670a-1114-4ab4-889b-0270257bb28c service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:10.026991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5670a-1114-4ab4-889b-0270257bb28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:10.086499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5670a-1114-4ab4-889b-0270257bb28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:10.087917 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5670a-1114-4ab4-889b-0270257bb28c service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:10.096766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1039/2088] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:16.028964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:16.091419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620310e7-ffab-4ff3-9330-b9ef2169377c service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:16.095926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:16.096583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:16.096866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:16.097142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] PATH_INFO: `/v3/users/6973b0b65c9b46fa8850e9920c9527ba` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:16.098251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6973b0b65c9b46fa8850e9920c9527ba)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:16.100187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:16.147083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6973b0b65c9b46fa8850e9920c9527ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.147736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6973b0b65c9b46fa8850e9920c9527ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.156486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6973b0b65c9b46fa8850e9920c9527ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.157169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6973b0b65c9b46fa8850e9920c9527ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.170321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6973b0b65c9b46fa8850e9920c9527ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.175445 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:16.183601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1050/2089] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:16 2022] DELETE /identity/v3/users/6973b0b65c9b46fa8850e9920c9527ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:16.188132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:16.188640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:16.188748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:16.206976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:16.208451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:16.269196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1040/2090] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:16.279217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:16.280657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:16.338457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded44d5-716a-4097-ab71-434425c97318 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:16.342571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:16.343176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:16.343460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:16.343731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:16.352544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:16.353975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:16.363805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1051/2091] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:16.649983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:16.651413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:16.665350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e6dfe-599e-4c2b-9dbf-d02776391d22 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:16.673015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:16.673670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:16.673948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:16.674221 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] PATH_INFO: `/v3/projects/389c509d7a7845f1a03bf6e0ae27e6cf` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:16.675350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=389c509d7a7845f1a03bf6e0ae27e6cf)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:16.677669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:16.730235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '389c509d7a7845f1a03bf6e0ae27e6cf', 'request_id': 'req-7065f58f-9d74-40db-8a92-cb0e93559967'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.736219 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '389c509d7a7845f1a03bf6e0ae27e6cf', 'request_id': 'req-7065f58f-9d74-40db-8a92-cb0e93559967'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.742525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '389c509d7a7845f1a03bf6e0ae27e6cf', 'request_id': 'req-7065f58f-9d74-40db-8a92-cb0e93559967'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.759253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] The token cache is being invalidate because project 389c509d7a7845f1a03bf6e0ae27e6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:16.759642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.760418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.761945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1041/2092] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:16 2022] DELETE /identity/v3/projects/389c509d7a7845f1a03bf6e0ae27e6cf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:57:16.765937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:16.775582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e945ff-fa44-4a99-b433-5230f89e4932 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:16.834842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:16.835481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:16.835760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:16.836039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] PATH_INFO: `/v3/users/21a9771ef36d4168a059ac1508c6ed73` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:16.837198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a9771ef36d4168a059ac1508c6ed73)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:16.839126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:16.896697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a9771ef36d4168a059ac1508c6ed73'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.897301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a9771ef36d4168a059ac1508c6ed73'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.905162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a9771ef36d4168a059ac1508c6ed73'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.905784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a9771ef36d4168a059ac1508c6ed73'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.917586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7ce2a57-c878-4809-a80f-51365ce63e5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a9771ef36d4168a059ac1508c6ed73'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:16.930185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1052/2093] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:16 2022] DELETE /identity/v3/users/21a9771ef36d4168a059ac1508c6ed73 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:17.201501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.203152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:17.216856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:17.222010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065f58f-9d74-40db-8a92-cb0e93559967 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:17.289055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:17.289690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.289973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.290480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] PATH_INFO: `/v3/projects/4e2be38532cc4090bf0ffec16c6c5332` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.292429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2be38532cc4090bf0ffec16c6c5332)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:17.294339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:17.343780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2be38532cc4090bf0ffec16c6c5332', 'request_id': 'req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:17.349345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2be38532cc4090bf0ffec16c6c5332', 'request_id': 'req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:17.355938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2be38532cc4090bf0ffec16c6c5332', 'request_id': 'req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:17.369624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] The token cache is being invalidate because project 4e2be38532cc4090bf0ffec16c6c5332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:17.369913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:17.370658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cc0dd6fe-a7f7-42f0-868b-b14d8773ff0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:17.372505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1042/2094] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:17 2022] DELETE /identity/v3/projects/4e2be38532cc4090bf0ffec16c6c5332 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:17.377351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4d3b-2ea4-44b4-9d7d-c4f0b0f650a7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.377889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4d3b-2ea4-44b4-9d7d-c4f0b0f650a7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.377889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4d3b-2ea4-44b4-9d7d-c4f0b0f650a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.397919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4caa4d3b-2ea4-44b4-9d7d-c4f0b0f650a7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:17.399129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4caa4d3b-2ea4-44b4-9d7d-c4f0b0f650a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:17.413158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:17.464208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1053/2095] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:17.472699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade02c0-8484-41e0-b355-94bd115cafe1 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.472992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade02c0-8484-41e0-b355-94bd115cafe1 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.473272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade02c0-8484-41e0-b355-94bd115cafe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.493014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8ade02c0-8484-41e0-b355-94bd115cafe1 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:17.494249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8ade02c0-8484-41e0-b355-94bd115cafe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:17.502177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1043/2096] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:17.507719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa4d3b-2ea4-44b4-9d7d-c4f0b0f650a7 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.517916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa4d3b-2ea4-44b4-9d7d-c4f0b0f650a7 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:17.522680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:17.523262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.523540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.523809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.524484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:17.524791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:17.526376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:17.538236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1054/2097] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:57:17.542423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade02c0-8484-41e0-b355-94bd115cafe1 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.551705 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade02c0-8484-41e0-b355-94bd115cafe1 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:17.556389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:17.556985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.557268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.557541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.558174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:17.560079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:17.588406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1044/2098] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:17 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:17.593265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.603048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758d3aa-c39b-40be-8be1-e057f1418762 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:17.660365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:17.660979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.661259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.661533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.662148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:17.664055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:17.687818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1055/2099] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:17 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:17.689008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.699506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e9972-c2ab-491c-adbb-c422aba7ed8f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:17.704949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:17.705564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.705844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.706149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.706793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:17.708703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:17.719140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1045/2100] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:57:17.723218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.733604 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33c7b-049b-424d-89a5-cc404869dc3a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:17.738433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f134-7b73-4f46-8f61-78c4d8cd5f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:17.739529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f134-7b73-4f46-8f61-78c4d8cd5f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.739529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f134-7b73-4f46-8f61-78c4d8cd5f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.739743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f134-7b73-4f46-8f61-78c4d8cd5f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.740265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0f134-7b73-4f46-8f61-78c4d8cd5f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:17.741888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0f134-7b73-4f46-8f61-78c4d8cd5f50 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:17.760321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1056/2101] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:57:17.761543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.772027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42e30e-0e5b-46e9-bbb7-387797a30fe7 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:17.776805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:17.777445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.777751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.778017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] PATH_INFO: `/v3/projects/7483e944593243cba9a37fe57e121061/users/604d08d126574d9888b4f99e41adf8f8/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.801151 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7483e944593243cba9a37fe57e121061, user_id=604d08d126574d9888b4f99e41adf8f8, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:17.804245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:17.819848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1046/2102] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:17 2022] PUT /identity/v3/projects/7483e944593243cba9a37fe57e121061/users/604d08d126574d9888b4f99e41adf8f8/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:57:17.831167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb6c3e-b730-4811-b419-357e3a2a10bf None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:17.831555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb6c3e-b730-4811-b419-357e3a2a10bf None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:17.831950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb6c3e-b730-4811-b419-357e3a2a10bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:17.867749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ccdb6c3e-b730-4811-b419-357e3a2a10bf None None] MFA Rules not processed for user `604d08d126574d9888b4f99e41adf8f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:17.868996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ccdb6c3e-b730-4811-b419-357e3a2a10bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:17.943220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1057/2103] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:17.947858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:17.988850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:18.030970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b92b-1d69-4668-97d6-bfee7a4200f3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:18.035707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:18.036355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.036637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.036918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.037580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:18.039492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:18.072298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1047/2104] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:17 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:18.076507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb6c3e-b730-4811-b419-357e3a2a10bf None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:18.089587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb6c3e-b730-4811-b419-357e3a2a10bf None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:18.110325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:18.171323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:18.171949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.172260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.172536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.173221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:18.175133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:18.202895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1058/2105] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:18 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:18.206967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:18.217950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f2e14-5739-43fb-9e9e-d191cc30ba93 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:18.223172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:18.223823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.224236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.224512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.225231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:18.226982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:18.239152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1048/2106] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:18.243593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:18.254393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cce42-5d72-414f-b8c1-1f99caec1712 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:18.259042 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:18.259867 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.259974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.260265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] PATH_INFO: `/v3/projects/4a456bbf72bc41ba846e85c1703ddbb8/users/0ec494d3294b4fa5a78966ec1fcaf5a4/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.294132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a456bbf72bc41ba846e85c1703ddbb8, user_id=0ec494d3294b4fa5a78966ec1fcaf5a4, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:18.296987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:18.312236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1059/2107] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:18 2022] PUT /identity/v3/projects/4a456bbf72bc41ba846e85c1703ddbb8/users/0ec494d3294b4fa5a78966ec1fcaf5a4/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:18.316310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:18.327915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595cea-44ac-4341-80b2-61a0df4f21dc admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:18.387883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201711bb-9f7a-4d5e-9a41-c17d227fa8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:18.388533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201711bb-9f7a-4d5e-9a41-c17d227fa8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.388822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201711bb-9f7a-4d5e-9a41-c17d227fa8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.389093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201711bb-9f7a-4d5e-9a41-c17d227fa8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.389747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201711bb-9f7a-4d5e-9a41-c17d227fa8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:18.391383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201711bb-9f7a-4d5e-9a41-c17d227fa8b9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:18.402000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1049/2108] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:18.406685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:18.423538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54117180-cf98-463e-9922-eeaad18eb85a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:18.428792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:18.429470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.429740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.430054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] PATH_INFO: `/v3/projects/4a456bbf72bc41ba846e85c1703ddbb8/users/0ec494d3294b4fa5a78966ec1fcaf5a4/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.431985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a456bbf72bc41ba846e85c1703ddbb8, user_id=0ec494d3294b4fa5a78966ec1fcaf5a4, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:18.434663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:18.454584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1060/2109] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:18 2022] PUT /identity/v3/projects/4a456bbf72bc41ba846e85c1703ddbb8/users/0ec494d3294b4fa5a78966ec1fcaf5a4/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:18.456140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd877bd-f450-4af0-a690-8d060bfa2c8f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.456462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd877bd-f450-4af0-a690-8d060bfa2c8f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.456767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd877bd-f450-4af0-a690-8d060bfa2c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.487329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3cd877bd-f450-4af0-a690-8d060bfa2c8f None None] MFA Rules not processed for user `0ec494d3294b4fa5a78966ec1fcaf5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:18.488515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3cd877bd-f450-4af0-a690-8d060bfa2c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:18.566535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1050/2110] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:18.577973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:18.581128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:18.643410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4005d-e793-4056-9836-fad053ad4be2 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:18.647082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af912b-4cef-4020-92ac-7469408e2af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:18.647694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af912b-4cef-4020-92ac-7469408e2af1 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:18.647977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af912b-4cef-4020-92ac-7469408e2af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:18.648285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af912b-4cef-4020-92ac-7469408e2af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:18.658015 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af912b-4cef-4020-92ac-7469408e2af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:18.659446 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af912b-4cef-4020-92ac-7469408e2af1 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:18.671230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1061/2111] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:19.073791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd877bd-f450-4af0-a690-8d060bfa2c8f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:19.105237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:19.138257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd877bd-f450-4af0-a690-8d060bfa2c8f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:19.142749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:19.143372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:19.143649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:19.143920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] PATH_INFO: `/v3/users/604d08d126574d9888b4f99e41adf8f8` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:19.145049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=604d08d126574d9888b4f99e41adf8f8)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:19.146967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:19.191233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604d08d126574d9888b4f99e41adf8f8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.198255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604d08d126574d9888b4f99e41adf8f8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.206581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604d08d126574d9888b4f99e41adf8f8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.207260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604d08d126574d9888b4f99e41adf8f8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.220092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604d08d126574d9888b4f99e41adf8f8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.226776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1051/2112] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:19 2022] DELETE /identity/v3/users/604d08d126574d9888b4f99e41adf8f8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:19.231194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df41098-a98b-4836-abe3-004514a6132d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:19.231484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df41098-a98b-4836-abe3-004514a6132d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:19.231862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df41098-a98b-4836-abe3-004514a6132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:19.251507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-1df41098-a98b-4836-abe3-004514a6132d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:19.253462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1df41098-a98b-4836-abe3-004514a6132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:19.320341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1062/2113] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:19.331077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:19.332623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:19.392528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21dd13-6f70-48fc-bb5f-705875970590 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:19.396462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:19.397070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:19.397349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:19.397622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:19.406841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:19.408313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:19.417810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1052/2114] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:19.699920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df41098-a98b-4836-abe3-004514a6132d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:19.701399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1df41098-a98b-4836-abe3-004514a6132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:19.713677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df41098-a98b-4836-abe3-004514a6132d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:19.718338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:19.718971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:19.719246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:19.719513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] PATH_INFO: `/v3/projects/7483e944593243cba9a37fe57e121061` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:19.720669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7483e944593243cba9a37fe57e121061)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:19.723040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:19.725396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:19.772028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7483e944593243cba9a37fe57e121061', 'request_id': 'req-b0c60b6f-5b73-4089-8cba-6dd57824c53f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.777504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7483e944593243cba9a37fe57e121061', 'request_id': 'req-b0c60b6f-5b73-4089-8cba-6dd57824c53f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.783749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7483e944593243cba9a37fe57e121061', 'request_id': 'req-b0c60b6f-5b73-4089-8cba-6dd57824c53f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.797503 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] The token cache is being invalidate because project 7483e944593243cba9a37fe57e121061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:19.797811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.798532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.800486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1063/2115] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:19 2022] DELETE /identity/v3/projects/7483e944593243cba9a37fe57e121061 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:19.804435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:19.814936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb353-6102-438c-a4b9-9e021f4d3f1b service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:19.869019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:19.883821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:19.884482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:19.884776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:19.885029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] PATH_INFO: `/v3/users/0ec494d3294b4fa5a78966ec1fcaf5a4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:19.886200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec494d3294b4fa5a78966ec1fcaf5a4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:19.888198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:19.936595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec494d3294b4fa5a78966ec1fcaf5a4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.943623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec494d3294b4fa5a78966ec1fcaf5a4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.950994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec494d3294b4fa5a78966ec1fcaf5a4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.951658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec494d3294b4fa5a78966ec1fcaf5a4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.963815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5b7d43f4-429d-42d2-9599-19f6275590e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec494d3294b4fa5a78966ec1fcaf5a4'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:19.970109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1053/2116] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:19 2022] DELETE /identity/v3/users/0ec494d3294b4fa5a78966ec1fcaf5a4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:20.254370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:20.255966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:20.270592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c60b6f-5b73-4089-8cba-6dd57824c53f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:20.328161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:20.328800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.329079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.329350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] PATH_INFO: `/v3/projects/4a456bbf72bc41ba846e85c1703ddbb8` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.330456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a456bbf72bc41ba846e85c1703ddbb8)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:20.332381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:20.383404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a456bbf72bc41ba846e85c1703ddbb8', 'request_id': 'req-74ac49da-b45e-472b-ac6e-c06e03e4eeae'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:20.390562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a456bbf72bc41ba846e85c1703ddbb8', 'request_id': 'req-74ac49da-b45e-472b-ac6e-c06e03e4eeae'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:20.398439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a456bbf72bc41ba846e85c1703ddbb8', 'request_id': 'req-74ac49da-b45e-472b-ac6e-c06e03e4eeae'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:20.420624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] The token cache is being invalidate because project 4a456bbf72bc41ba846e85c1703ddbb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:20.420926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:20.421574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-74ac49da-b45e-472b-ac6e-c06e03e4eeae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:20.424452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1064/2117] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:20 2022] DELETE /identity/v3/projects/4a456bbf72bc41ba846e85c1703ddbb8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:20.429871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ea82f-9490-446a-898d-c574da4228a4 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.431331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ea82f-9490-446a-898d-c574da4228a4 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.431850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ea82f-9490-446a-898d-c574da4228a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.457541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-545ea82f-9490-446a-898d-c574da4228a4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:20.459057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-545ea82f-9490-446a-898d-c574da4228a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:20.551506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1054/2118] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:20.561532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcac83f-49fd-4d9e-a4bc-9bfd278b5eaf None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.562107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcac83f-49fd-4d9e-a4bc-9bfd278b5eaf None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.562466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcac83f-49fd-4d9e-a4bc-9bfd278b5eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.585491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-cbcac83f-49fd-4d9e-a4bc-9bfd278b5eaf None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:20.586769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-cbcac83f-49fd-4d9e-a4bc-9bfd278b5eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:20.596653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1065/2119] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:20.601445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ea82f-9490-446a-898d-c574da4228a4 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:20.611856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ea82f-9490-446a-898d-c574da4228a4 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:20.616763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:20.617364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.617641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.617910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.618550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:20.618849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:20.620461 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:20.635975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1055/2120] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:57:20.638690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcac83f-49fd-4d9e-a4bc-9bfd278b5eaf None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:20.649743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcac83f-49fd-4d9e-a4bc-9bfd278b5eaf None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:20.655245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:20.655866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.656180 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.656457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.657092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:20.659302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:20.674456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:20.689696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1066/2121] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:20 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:20.693998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:20.704472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b18b-0096-4ae8-82bb-ceafa60cb453 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:20.763729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:20.772059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:20.772877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.773499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.773892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.774960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:20.777005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:20.799764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1056/2122] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:20 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:20.804224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:20.815397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638366ce-491a-4cd7-abd6-05f2a53c6180 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:20.823506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65942-d659-432a-9727-61484970395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:20.824211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65942-d659-432a-9727-61484970395b admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.824598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65942-d659-432a-9727-61484970395b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.824902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65942-d659-432a-9727-61484970395b admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.825568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d65942-d659-432a-9727-61484970395b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:20.827729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d65942-d659-432a-9727-61484970395b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:20.839610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1067/2123] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:20.845790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:20.856518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06c386-31c9-4c4b-8dc8-4db29b033223 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:20.861248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f140e3c-704b-4ba8-a94f-6cbd64da5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:20.861859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f140e3c-704b-4ba8-a94f-6cbd64da5282 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.862142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f140e3c-704b-4ba8-a94f-6cbd64da5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.862418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f140e3c-704b-4ba8-a94f-6cbd64da5282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.863060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f140e3c-704b-4ba8-a94f-6cbd64da5282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:20.864711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f140e3c-704b-4ba8-a94f-6cbd64da5282 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:20.875333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1057/2124] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:20.879478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65942-d659-432a-9727-61484970395b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:20.889563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65942-d659-432a-9727-61484970395b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:20.894239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:20.894892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.895174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.895450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] PATH_INFO: `/v3/projects/9f56c321a8fd4ab39bff7b75caa8d2f8/users/6cdc87afe77a4d229579a5ab5c1486ea/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.910423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f56c321a8fd4ab39bff7b75caa8d2f8, user_id=6cdc87afe77a4d229579a5ab5c1486ea, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:20.913132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:20.927107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1068/2125] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:20 2022] PUT /identity/v3/projects/9f56c321a8fd4ab39bff7b75caa8d2f8/users/6cdc87afe77a4d229579a5ab5c1486ea/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:20.937317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee667d4-dd87-4ce5-b05e-3c90cbe24da8 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:20.937610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee667d4-dd87-4ce5-b05e-3c90cbe24da8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:20.937891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee667d4-dd87-4ce5-b05e-3c90cbe24da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:20.965717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-0ee667d4-dd87-4ce5-b05e-3c90cbe24da8 None None] MFA Rules not processed for user `6cdc87afe77a4d229579a5ab5c1486ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:20.966931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0ee667d4-dd87-4ce5-b05e-3c90cbe24da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:21.032603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1058/2126] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:21.044354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:21.046057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:21.106251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f186a-fa92-4cf0-b6bf-e45cb70436d5 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:21.116457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633ef3c-fb69-4aca-a383-261c15551735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:21.116457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633ef3c-fb69-4aca-a383-261c15551735 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.116457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633ef3c-fb69-4aca-a383-261c15551735 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.116457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633ef3c-fb69-4aca-a383-261c15551735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.119649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633ef3c-fb69-4aca-a383-261c15551735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:21.121135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633ef3c-fb69-4aca-a383-261c15551735 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:21.135390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1069/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:21.157345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee667d4-dd87-4ce5-b05e-3c90cbe24da8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:21.248861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee667d4-dd87-4ce5-b05e-3c90cbe24da8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:21.256198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:21.256838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.257119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.257394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] PATH_INFO: `/v3/users/6cdc87afe77a4d229579a5ab5c1486ea` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.263259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdc87afe77a4d229579a5ab5c1486ea)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:21.265392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:21.317005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdc87afe77a4d229579a5ab5c1486ea'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.324001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdc87afe77a4d229579a5ab5c1486ea'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.331466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdc87afe77a4d229579a5ab5c1486ea'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.332086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdc87afe77a4d229579a5ab5c1486ea'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.334268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:21.348421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdc87afe77a4d229579a5ab5c1486ea'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.354591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1059/2128] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:21 2022] DELETE /identity/v3/users/6cdc87afe77a4d229579a5ab5c1486ea => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:21.359043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.359338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.359620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.377935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:21.379141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:21.438624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1070/2129] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:21.449230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:21.451364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:21.507600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70599ec6-d8be-414c-b2b9-084c8295e2d0 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:21.511221 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:21.511815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.512739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.513014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.522572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:21.524030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:21.534338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1060/2130] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:21.818940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:21.820398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:21.833918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b4f16-89b3-4561-a58f-e703ce1603d0 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:21.839284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:21.840326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.840448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:21.840639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.840880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] PATH_INFO: `/v3/users/073cab5629fa4d3c8f5bd6bfe0689976` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.841966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:21.842083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073cab5629fa4d3c8f5bd6bfe0689976)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:21.843927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:21.855534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b3dc-98c1-459b-bd21-c9b6a5dc7faf service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:21.860782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:21.861444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.861700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.861973 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] PATH_INFO: `/v3/projects/9f56c321a8fd4ab39bff7b75caa8d2f8` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.863097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f56c321a8fd4ab39bff7b75caa8d2f8)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:21.865057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:21.901395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073cab5629fa4d3c8f5bd6bfe0689976'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.902143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073cab5629fa4d3c8f5bd6bfe0689976'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.917153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f56c321a8fd4ab39bff7b75caa8d2f8', 'request_id': 'req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.917544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073cab5629fa4d3c8f5bd6bfe0689976'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.918800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073cab5629fa4d3c8f5bd6bfe0689976'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.921335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:21.924174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f56c321a8fd4ab39bff7b75caa8d2f8', 'request_id': 'req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.931741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f56c321a8fd4ab39bff7b75caa8d2f8', 'request_id': 'req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.935608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07db3a74-9ec4-4c8a-a771-75f759c05f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073cab5629fa4d3c8f5bd6bfe0689976'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.945451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] The token cache is being invalidate because project 9f56c321a8fd4ab39bff7b75caa8d2f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:21.945755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.946377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a3197c22-6d3c-44d9-a2ef-2a7f7b1813ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:21.948137 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1061/2131] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:21 2022] DELETE /identity/v3/projects/9f56c321a8fd4ab39bff7b75caa8d2f8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:21.948895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1071/2132] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:21 2022] DELETE /identity/v3/users/073cab5629fa4d3c8f5bd6bfe0689976 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:21.953306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf679d2-f7d3-44d3-b1c5-e0aa1569114f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.953542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:21.953646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf679d2-f7d3-44d3-b1c5-e0aa1569114f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.953837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:21.953949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf679d2-f7d3-44d3-b1c5-e0aa1569114f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.954252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:21.974937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2bf679d2-f7d3-44d3-b1c5-e0aa1569114f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:21.976138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2bf679d2-f7d3-44d3-b1c5-e0aa1569114f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:21.978281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:21.980179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.041033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1062/2133] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:22.048099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1072/2134] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:22.049640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599521f3-bbed-47b7-882c-73d117eaaad7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.049965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599521f3-bbed-47b7-882c-73d117eaaad7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.050345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599521f3-bbed-47b7-882c-73d117eaaad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.057525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.059335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.075673 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-599521f3-bbed-47b7-882c-73d117eaaad7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:22.076905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-599521f3-bbed-47b7-882c-73d117eaaad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.085144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1063/2135] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:22.089409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599521f3-bbed-47b7-882c-73d117eaaad7 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.093670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:22.098722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599521f3-bbed-47b7-882c-73d117eaaad7 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.105344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.106597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.107041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.107503 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.108181 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:22.108482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.110087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.118636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0090ae8-0e23-43f5-87dd-743bb90165b1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.121832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1064/2136] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:57:22.126344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.126502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.126966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.127356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.127611 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.136726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c8210-2e11-4645-bf7e-9f1fdebb195a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.142095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.142731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.143016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.143296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.143935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.144331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.145909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.146165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.164352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1073/2137] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:22.175543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1065/2138] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:22.179662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.190028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02439616-0106-4770-9c21-84f40a72cc55 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.254149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.254767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.255046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.255961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.256636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.258545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.283761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1074/2139] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/users => generated 281 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:22.285297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.296391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671233-12bc-407c-8885-296b3c8b211a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.307543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.308307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.308680 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.309548 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.310351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.312180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.322965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1066/2140] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:57:22.327569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.338176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2062c-1826-4f70-a9d5-943ed2f11545 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.343610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3cd8b-8e9a-4401-96f0-790f571a6286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.344263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3cd8b-8e9a-4401-96f0-790f571a6286 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.344545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3cd8b-8e9a-4401-96f0-790f571a6286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.344814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3cd8b-8e9a-4401-96f0-790f571a6286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.345456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3cd8b-8e9a-4401-96f0-790f571a6286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.347046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3cd8b-8e9a-4401-96f0-790f571a6286 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.348418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:22.358546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1075/2141] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:22.362570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.373628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20656b8-5262-4cfc-88e8-aa972946a437 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.378763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.379514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.379776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.379990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] PATH_INFO: `/v3/projects/9fcb36a7bed54092b39a8f56eee8c1a5/users/b7e3c545e5c6427ba32b7bef6d438a92/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.395982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcb36a7bed54092b39a8f56eee8c1a5, user_id=b7e3c545e5c6427ba32b7bef6d438a92, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.398696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.414991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1067/2142] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:22 2022] PUT /identity/v3/projects/9fcb36a7bed54092b39a8f56eee8c1a5/users/b7e3c545e5c6427ba32b7bef6d438a92/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:22.430249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.430650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.431028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.444387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.446163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.462414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] MFA Rules not processed for user `b7e3c545e5c6427ba32b7bef6d438a92`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:22.463641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.517363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a897c-0a7e-400f-a43c-ef7703d179ce admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.522810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.523456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.523738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.524012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] PATH_INFO: `/v3/projects/dd5678401aa84013b01e2686614643ef` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.525194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd5678401aa84013b01e2686614643ef)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.527103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.543704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1076/2143] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:22.577681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5678401aa84013b01e2686614643ef', 'request_id': 'req-efcbba47-9f6e-4276-a87a-9938d38fd4ee'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.578636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.580558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.583955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5678401aa84013b01e2686614643ef', 'request_id': 'req-efcbba47-9f6e-4276-a87a-9938d38fd4ee'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.591185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5678401aa84013b01e2686614643ef', 'request_id': 'req-efcbba47-9f6e-4276-a87a-9938d38fd4ee'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.609116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] The token cache is being invalidate because project dd5678401aa84013b01e2686614643ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:22.609384 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.609974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-efcbba47-9f6e-4276-a87a-9938d38fd4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.611751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1068/2144] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:22 2022] DELETE /identity/v3/projects/dd5678401aa84013b01e2686614643ef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:57:22.618214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92582593-c390-494a-ac7e-fdd85fd5d7d3 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.618646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92582593-c390-494a-ac7e-fdd85fd5d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.619066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92582593-c390-494a-ac7e-fdd85fd5d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.644800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-92582593-c390-494a-ac7e-fdd85fd5d7d3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:22.647188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-92582593-c390-494a-ac7e-fdd85fd5d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.658736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef501cef-5501-4d88-955b-9467a692c306 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.666586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:22.711471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.712088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.712380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.712658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.714487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.714596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1069/2145] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:22.722319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a25f9-09f4-4642-abdf-ed1b4635b84b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.722605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a25f9-09f4-4642-abdf-ed1b4635b84b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.722879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a25f9-09f4-4642-abdf-ed1b4635b84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.744495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-449a25f9-09f4-4642-abdf-ed1b4635b84b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:22.745718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-449a25f9-09f4-4642-abdf-ed1b4635b84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.754451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1070/2146] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:22.758703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a25f9-09f4-4642-abdf-ed1b4635b84b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.767557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a25f9-09f4-4642-abdf-ed1b4635b84b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.772446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.772768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:22.773043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.773330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.773623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.774387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:22.774753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.775354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.776560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.777185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.790100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1071/2147] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:57:22.790685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1077/2148] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:22.794553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.804633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bb1f1-fb64-4f2a-87fb-e2298fc3b974 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.806555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.807935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.809651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.810249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.810524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.810794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.811415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.814615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.822161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb1694-5b84-47d4-adec-d973dc2091d6 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.827708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.828518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.828820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.829107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] PATH_INFO: `/v3/users/b7e3c545e5c6427ba32b7bef6d438a92` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.830205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e3c545e5c6427ba32b7bef6d438a92)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.832164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.846782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1072/2149] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:22.850936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.889645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e3c545e5c6427ba32b7bef6d438a92'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.890605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e3c545e5c6427ba32b7bef6d438a92'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.900105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e3c545e5c6427ba32b7bef6d438a92'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.900882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e3c545e5c6427ba32b7bef6d438a92'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.920698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bf000b5f-2791-4746-a1de-8db6be5c3d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e3c545e5c6427ba32b7bef6d438a92'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:22.931033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866df56f-9877-4d0c-a346-2e27103d548a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:22.933526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1078/2150] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:22 2022] DELETE /identity/v3/users/b7e3c545e5c6427ba32b7bef6d438a92 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:22.937042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:22.937677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.937970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.938240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.938346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:22.939237 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:22.939351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:22.939451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:22.941283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:22.959561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:22.960961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.970472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1073/2151] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:22.974629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:22.976964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:22.989830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb900f5-2f41-4c41-b646-5cdcc11975bb admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.044308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1079/2152] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:23.052336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.053931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:23.071098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.071835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.072118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.072438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.073117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.074817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.086419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1074/2153] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:23.090827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.101373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080aa87-8a2e-4f38-bdf8-e5c794619b31 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.107158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.107761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.108204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.108469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.109149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.110819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.112378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:23.122401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc2e47-dc86-4ff9-bbd8-adf81dcbee7d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.127328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1075/2154] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:23.128541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.129341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc52df-8549-4f0d-af41-b018f992fb17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.129974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc52df-8549-4f0d-af41-b018f992fb17 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.130241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc52df-8549-4f0d-af41-b018f992fb17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.130514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc52df-8549-4f0d-af41-b018f992fb17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.138346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a8e6f-5243-451d-9fe8-c45dd74bbe6e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.139911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc52df-8549-4f0d-af41-b018f992fb17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.141392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc52df-8549-4f0d-af41-b018f992fb17 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.143182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.143849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.144128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.144432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] PATH_INFO: `/v3/projects/1bfcc2ab05ef477aba1a631fe2b799ce/users/7b731562ef8c415fbb0c205cd1fe84d0/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.151016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1080/2155] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:23.159767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfcc2ab05ef477aba1a631fe2b799ce, user_id=7b731562ef8c415fbb0c205cd1fe84d0, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.162644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.176709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1076/2156] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:23 2022] PUT /identity/v3/projects/1bfcc2ab05ef477aba1a631fe2b799ce/users/7b731562ef8c415fbb0c205cd1fe84d0/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:57:23.187836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.188118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.188465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.210653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:23.230167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] MFA Rules not processed for user `7b731562ef8c415fbb0c205cd1fe84d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:23.233809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:23.306283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1081/2157] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:23.317529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.319846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:23.385213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140d757-c662-4121-ba45-cd55f69fd146 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.389560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c41aa-db69-4d81-a8a8-d840f6b2645f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.390224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c41aa-db69-4d81-a8a8-d840f6b2645f service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.390443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c41aa-db69-4d81-a8a8-d840f6b2645f service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.390710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c41aa-db69-4d81-a8a8-d840f6b2645f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.399554 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c41aa-db69-4d81-a8a8-d840f6b2645f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.401166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c41aa-db69-4d81-a8a8-d840f6b2645f service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.424585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1077/2158] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:23.448283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.450350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:23.522620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9df640-7b35-4c83-8e1a-0cb847a9d671 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.527382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.528639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.528957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.529231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] PATH_INFO: `/v3/projects/9fcb36a7bed54092b39a8f56eee8c1a5` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.530364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcb36a7bed54092b39a8f56eee8c1a5)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.532313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.591426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcb36a7bed54092b39a8f56eee8c1a5', 'request_id': 'req-05137e15-9a1f-4bd0-9ca8-584bb4699034'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:23.597969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcb36a7bed54092b39a8f56eee8c1a5', 'request_id': 'req-05137e15-9a1f-4bd0-9ca8-584bb4699034'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:23.604376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcb36a7bed54092b39a8f56eee8c1a5', 'request_id': 'req-05137e15-9a1f-4bd0-9ca8-584bb4699034'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:23.619101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] The token cache is being invalidate because project 9fcb36a7bed54092b39a8f56eee8c1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:23.619388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:23.620084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-05137e15-9a1f-4bd0-9ca8-584bb4699034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:23.621956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1082/2159] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:23 2022] DELETE /identity/v3/projects/9fcb36a7bed54092b39a8f56eee8c1a5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:23.627236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f5be-9e5a-41ec-83d5-126fa565f777 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.627876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f5be-9e5a-41ec-83d5-126fa565f777 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.627876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f5be-9e5a-41ec-83d5-126fa565f777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.648408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5e63f5be-9e5a-41ec-83d5-126fa565f777 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:23.649561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5e63f5be-9e5a-41ec-83d5-126fa565f777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:23.721473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1078/2160] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:23.729627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2e789-dc86-4625-97f1-a9503594da6d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.729913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2e789-dc86-4625-97f1-a9503594da6d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.730191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2e789-dc86-4625-97f1-a9503594da6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.752421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-1de2e789-dc86-4625-97f1-a9503594da6d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:23.754359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1de2e789-dc86-4625-97f1-a9503594da6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:23.764693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1083/2161] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 16:57:23.769027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f5be-9e5a-41ec-83d5-126fa565f777 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.778787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f5be-9e5a-41ec-83d5-126fa565f777 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.783527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.784201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.784462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.784735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.785392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:23.785693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.787287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.799394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1079/2162] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:57:23.803645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2e789-dc86-4625-97f1-a9503594da6d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.813411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2e789-dc86-4625-97f1-a9503594da6d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.818576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.819184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.819506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.819778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.820468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.822395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.849514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1084/2163] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:23 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:23.853618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.863471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197a7fa-755d-4c29-be43-a14cefda5663 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.920847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.921353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.921633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.921908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.922591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.924549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.933149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:23.946385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1080/2164] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:23 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:23.951171 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:23.956710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:23.963331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa08c-815f-4f12-87d1-bf24c98c4ffd admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:23.971033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:23.972868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:23.972868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:23.972868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:23.973142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:23.975742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:23.986528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1085/2165] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:57:23.990559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:24.001452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71132ae0-3052-49f7-a284-044fcbe8fc8a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:24.006419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429e3a4-d42b-4a89-9972-84221564ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:24.007015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e3a4-d42b-4a89-9972-84221564ff7f admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:24.007300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e3a4-d42b-4a89-9972-84221564ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:24.007576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e3a4-d42b-4a89-9972-84221564ff7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:24.008251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429e3a4-d42b-4a89-9972-84221564ff7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:24.009953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429e3a4-d42b-4a89-9972-84221564ff7f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:24.024664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1081/2166] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:24.025819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:24.036884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f281f7-3754-4498-88e1-55b30e5b9754 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:24.041946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:24.042608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:24.042888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:24.043654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] PATH_INFO: `/v3/projects/abe3003a0c91471e9982327ac5e17b9e/users/5d1f341b9aff4b7db13e429b86ba145e/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:24.059864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe3003a0c91471e9982327ac5e17b9e, user_id=5d1f341b9aff4b7db13e429b86ba145e, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:24.062559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:24.080777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1086/2167] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:24 2022] PUT /identity/v3/projects/abe3003a0c91471e9982327ac5e17b9e/users/5d1f341b9aff4b7db13e429b86ba145e/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:24.090923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9b96a-34e3-4283-a81f-0a0ab678c5d3 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:24.090923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9b96a-34e3-4283-a81f-0a0ab678c5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:24.090923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9b96a-34e3-4283-a81f-0a0ab678c5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:24.112428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-25c9b96a-34e3-4283-a81f-0a0ab678c5d3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:24.113775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-25c9b96a-34e3-4283-a81f-0a0ab678c5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:24.179453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1082/2168] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:24.187590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:24.189340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:24.244819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6ba58-4aa4-48d7-a75e-f897fb0d4a63 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:24.249550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:24.250170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:24.250464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:24.250799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:24.266537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:24.268004 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:24.277908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1087/2169] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:24.928138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d90b-8c06-40f1-a666-162df8787634 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:24.929061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d90b-8c06-40f1-a666-162df8787634 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:24.929361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d90b-8c06-40f1-a666-162df8787634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:24.958779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-0e29d90b-8c06-40f1-a666-162df8787634 None None] MFA Rules not processed for user `5d1f341b9aff4b7db13e429b86ba145e`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:24.959899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0e29d90b-8c06-40f1-a666-162df8787634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:25.039587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1083/2170] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:25.051551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:25.053337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:25.115923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637a4d-b81f-4893-883b-1cbd2674e516 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:25.119864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:25.120530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:25.120814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:25.121089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:25.130780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:25.132557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:25.147957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1088/2171] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:27.584250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0168fb2-8b41-4587-ba2f-79be0d467986 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:27.584772 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0168fb2-8b41-4587-ba2f-79be0d467986 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:27.584878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0168fb2-8b41-4587-ba2f-79be0d467986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:27.603730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e0168fb2-8b41-4587-ba2f-79be0d467986 None None] MFA Rules not processed for user `5d1f341b9aff4b7db13e429b86ba145e`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:27.604972 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e0168fb2-8b41-4587-ba2f-79be0d467986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:27.611933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1084/2172] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:35.143490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:37.598177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:INSERT=2,keystone:SELECT=69 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:40.287906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:40.289805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:40.310409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ddca-12bb-45ec-8532-917c5a939faa service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:40.316586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:40.317491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:40.317900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:40.318458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] PATH_INFO: `/v3/users/7b731562ef8c415fbb0c205cd1fe84d0` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:40.319330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b731562ef8c415fbb0c205cd1fe84d0)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:40.321308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:40.382487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b731562ef8c415fbb0c205cd1fe84d0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:40.383166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b731562ef8c415fbb0c205cd1fe84d0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:40.393100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b731562ef8c415fbb0c205cd1fe84d0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:40.393954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b731562ef8c415fbb0c205cd1fe84d0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:40.412374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b731562ef8c415fbb0c205cd1fe84d0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:40.428188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1089/2173] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:40 2022] DELETE /identity/v3/users/7b731562ef8c415fbb0c205cd1fe84d0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:40.429780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:40.430223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:40.430325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:40.449586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:40.451427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:40.513795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1085/2174] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:40.528871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:40.530397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:40.598415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d296f-0a63-4b4d-9f65-0bfe3df5db78 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:40.602288 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f216a9-5b38-4eee-bfd3-98314d3852dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:40.602921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f216a9-5b38-4eee-bfd3-98314d3852dd service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:40.603205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f216a9-5b38-4eee-bfd3-98314d3852dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:40.603483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f216a9-5b38-4eee-bfd3-98314d3852dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:40.619117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f216a9-5b38-4eee-bfd3-98314d3852dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:40.620633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f216a9-5b38-4eee-bfd3-98314d3852dd service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:40.633285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1090/2175] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:40.926165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:40.927662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:40.942626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2bd92-5845-48c9-ad4e-cf76f71d0fe9 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:40.950302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:40.951114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:40.951114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:40.951513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] PATH_INFO: `/v3/projects/1bfcc2ab05ef477aba1a631fe2b799ce` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:40.952746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfcc2ab05ef477aba1a631fe2b799ce)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:40.955204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.006325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfcc2ab05ef477aba1a631fe2b799ce', 'request_id': 'req-a7df0a20-01e6-487f-9796-a78847a3d30d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:41.011866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfcc2ab05ef477aba1a631fe2b799ce', 'request_id': 'req-a7df0a20-01e6-487f-9796-a78847a3d30d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:41.018047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfcc2ab05ef477aba1a631fe2b799ce', 'request_id': 'req-a7df0a20-01e6-487f-9796-a78847a3d30d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:41.030944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] The token cache is being invalidate because project 1bfcc2ab05ef477aba1a631fe2b799ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:41.031220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:41.031827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a7df0a20-01e6-487f-9796-a78847a3d30d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:41.033378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1086/2176] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:40 2022] DELETE /identity/v3/projects/1bfcc2ab05ef477aba1a631fe2b799ce => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:41.038267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33beedd3-da26-46f3-9436-e45483f2d03e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.038601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33beedd3-da26-46f3-9436-e45483f2d03e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.038874 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33beedd3-da26-46f3-9436-e45483f2d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.058488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-33beedd3-da26-46f3-9436-e45483f2d03e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:41.059726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-33beedd3-da26-46f3-9436-e45483f2d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:41.117186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1091/2177] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:41.126212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67ddd-b01c-49aa-b41f-8da6821d1240 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.126508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67ddd-b01c-49aa-b41f-8da6821d1240 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.126787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67ddd-b01c-49aa-b41f-8da6821d1240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.147088 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d4b67ddd-b01c-49aa-b41f-8da6821d1240 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:41.148383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d4b67ddd-b01c-49aa-b41f-8da6821d1240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:41.157199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1087/2178] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:41.161817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33beedd3-da26-46f3-9436-e45483f2d03e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:41.172210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33beedd3-da26-46f3-9436-e45483f2d03e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:41.176886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:41.177496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.177775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.178048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.178699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:41.179006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:41.180641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.192436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1092/2179] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:57:41.196616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b67ddd-b01c-49aa-b41f-8da6821d1240 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:41.206144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b67ddd-b01c-49aa-b41f-8da6821d1240 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:41.211013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:41.211605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.211911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.212210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.212829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:41.214718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.243849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1088/2180] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:41 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:41.247050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:41.252526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:41.259182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c8da4-4420-491e-87c4-3f90a0314775 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:41.315575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:41.316237 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.316482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.316760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.317376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:41.319323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.341406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1093/2181] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:41 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:57:41.345606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:41.356264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d31e3-879e-45b7-849c-e13700ab0e37 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:41.360803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:41.361397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.361676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.361946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.362583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:41.364223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.374016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1089/2182] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:41.377777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:41.387970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf457c1-591e-408c-9a73-99bfbf1cc3eb admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:41.392836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1c414-7bfa-46fa-93f0-f4cb34078cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:41.393420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c414-7bfa-46fa-93f0-f4cb34078cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.393696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c414-7bfa-46fa-93f0-f4cb34078cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.393964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c414-7bfa-46fa-93f0-f4cb34078cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.394595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1c414-7bfa-46fa-93f0-f4cb34078cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:41.396186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1c414-7bfa-46fa-93f0-f4cb34078cb5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.406943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1094/2183] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:57:41.410902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:41.421090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28118797-b115-44ba-9871-f4d1058fb9bf admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:41.425766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:41.426456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.426747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.427133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] PATH_INFO: `/v3/projects/9f4d5e97e38a4a11bdc073f8bd265438/users/1fea695ec966462692bb378816bee580/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.441649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4d5e97e38a4a11bdc073f8bd265438, user_id=1fea695ec966462692bb378816bee580, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:41.444368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.459116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1090/2184] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:41 2022] PUT /identity/v3/projects/9f4d5e97e38a4a11bdc073f8bd265438/users/1fea695ec966462692bb378816bee580/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:41.470075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb3f3b-edee-4b87-9bfb-3e060bac8543 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.470364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb3f3b-edee-4b87-9bfb-3e060bac8543 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.470633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb3f3b-edee-4b87-9bfb-3e060bac8543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.497422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-fdcb3f3b-edee-4b87-9bfb-3e060bac8543 None None] MFA Rules not processed for user `1fea695ec966462692bb378816bee580`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:41.498634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fdcb3f3b-edee-4b87-9bfb-3e060bac8543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:41.564902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1095/2185] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:41.576666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:41.578320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:41.601094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:41.639916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc585cb1-4107-4267-b2b4-ae58e39ffc9a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:41.643636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25182b-4b85-4079-8400-330bf8a10ae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:41.644287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25182b-4b85-4079-8400-330bf8a10ae0 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:41.644568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25182b-4b85-4079-8400-330bf8a10ae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:41.644838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25182b-4b85-4079-8400-330bf8a10ae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:41.654243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25182b-4b85-4079-8400-330bf8a10ae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:41.655708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25182b-4b85-4079-8400-330bf8a10ae0 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:41.665754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1091/2186] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:51.561934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:51.660977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=17 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:52.918348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb3f3b-edee-4b87-9bfb-3e060bac8543 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:52.991430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb3f3b-edee-4b87-9bfb-3e060bac8543 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:52.995873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:52.996536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:52.997337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:52.997609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] PATH_INFO: `/v3/users/1fea695ec966462692bb378816bee580` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:52.998812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fea695ec966462692bb378816bee580)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:53.000745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:53.050118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fea695ec966462692bb378816bee580'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.050921 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fea695ec966462692bb378816bee580'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.061553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fea695ec966462692bb378816bee580'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.062341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fea695ec966462692bb378816bee580'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.077047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fea695ec966462692bb378816bee580'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.090893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1096/2187] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:52 2022] DELETE /identity/v3/users/1fea695ec966462692bb378816bee580 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:53.095411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:53.095902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:53.095984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:53.114847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:53.116051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:53.182936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1092/2188] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:53.194120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:53.195692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:53.274417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f2da7-9d34-4bfc-9f2e-6aa91a00d0c1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:53.278268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17bec3-65ff-4c84-9f65-39543e9be293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:53.278942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17bec3-65ff-4c84-9f65-39543e9be293 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:53.279240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17bec3-65ff-4c84-9f65-39543e9be293 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:53.279460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17bec3-65ff-4c84-9f65-39543e9be293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:53.292500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17bec3-65ff-4c84-9f65-39543e9be293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:53.294495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17bec3-65ff-4c84-9f65-39543e9be293 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:53.310719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1097/2189] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:53.524200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:53.525803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:53.538446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e33bdc-12e5-481d-b824-477c37e29793 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:53.543143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:53.544407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:53.544789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:53.545282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] PATH_INFO: `/v3/users/f6b39e5a8d2042b5b36fde55301dec25` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:53.546530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b39e5a8d2042b5b36fde55301dec25)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:53.548658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:53.617710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b39e5a8d2042b5b36fde55301dec25'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.628008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b39e5a8d2042b5b36fde55301dec25'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.639188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b39e5a8d2042b5b36fde55301dec25'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.640021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b39e5a8d2042b5b36fde55301dec25'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.658864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b39e5a8d2042b5b36fde55301dec25'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.667992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1093/2190] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:57:53 2022] DELETE /identity/v3/users/f6b39e5a8d2042b5b36fde55301dec25 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:53.673331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:53.673833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:53.674117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:53.685736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:53.687460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:53.694464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:53.701073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:53.723111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:53.766966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce4a93-141b-405b-a5c2-eeaf93f92d0a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:53.768895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1098/2191] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:53.772803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:53.773459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:53.773740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:53.774013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] PATH_INFO: `/v3/projects/9f4d5e97e38a4a11bdc073f8bd265438` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:53.775191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4d5e97e38a4a11bdc073f8bd265438)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:53.777290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:53.779941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:53.781407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:53.800663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:53.847805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4d5e97e38a4a11bdc073f8bd265438', 'request_id': 'req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.853671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4d5e97e38a4a11bdc073f8bd265438', 'request_id': 'req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.860659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4d5e97e38a4a11bdc073f8bd265438', 'request_id': 'req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.861351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96945-52c7-4a19-bd1f-fbd5fed2b720 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:53.874186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] The token cache is being invalidate because project 9f4d5e97e38a4a11bdc073f8bd265438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:53.874472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.875328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:53.876858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1094/2192] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:53 2022] DELETE /identity/v3/projects/9f4d5e97e38a4a11bdc073f8bd265438 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:57:53.920508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53386844-90f9-49b8-b5bb-da07e99490e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:53.921145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53386844-90f9-49b8-b5bb-da07e99490e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:53.921424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53386844-90f9-49b8-b5bb-da07e99490e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:53.921699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53386844-90f9-49b8-b5bb-da07e99490e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:53.923800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-53386844-90f9-49b8-b5bb-da07e99490e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:54.025560 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53386844-90f9-49b8-b5bb-da07e99490e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.030650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53386844-90f9-49b8-b5bb-da07e99490e4 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.046386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1099/2193] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 16:57:54.406681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:54.408377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:54.428083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b27ba-fd82-43a0-9e35-dc5a82e8b8fa admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:54.433871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:54.434553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.434827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.435098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] PATH_INFO: `/v3/projects/ddb8a4af246141779f34d908e3a1de00` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.436252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb8a4af246141779f34d908e3a1de00)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.438853 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.488891 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb8a4af246141779f34d908e3a1de00', 'request_id': 'req-955fca0d-10c7-40fc-b1db-f1f141ad5e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:54.495593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb8a4af246141779f34d908e3a1de00', 'request_id': 'req-955fca0d-10c7-40fc-b1db-f1f141ad5e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:54.502705 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb8a4af246141779f34d908e3a1de00', 'request_id': 'req-955fca0d-10c7-40fc-b1db-f1f141ad5e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:54.515860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] The token cache is being invalidate because project ddb8a4af246141779f34d908e3a1de00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:57:54.516172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:54.516797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-955fca0d-10c7-40fc-b1db-f1f141ad5e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:57:54.518281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1095/2194] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:57:54 2022] DELETE /identity/v3/projects/ddb8a4af246141779f34d908e3a1de00 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:54.524204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9177c-c08c-488b-b2c4-53fcbc62e585 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.524512 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9177c-c08c-488b-b2c4-53fcbc62e585 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.524789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9177c-c08c-488b-b2c4-53fcbc62e585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.545284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a4e9177c-c08c-488b-b2c4-53fcbc62e585 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:54.546547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a4e9177c-c08c-488b-b2c4-53fcbc62e585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:54.570189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:54.605919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1100/2195] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:54.615404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fcdd-436c-4fe4-81bf-70bb015486ee None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.615693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fcdd-436c-4fe4-81bf-70bb015486ee None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.615971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fcdd-436c-4fe4-81bf-70bb015486ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.635289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d0b1fcdd-436c-4fe4-81bf-70bb015486ee None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:54.636540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d0b1fcdd-436c-4fe4-81bf-70bb015486ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:54.644806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1096/2196] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:54.648722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9177c-c08c-488b-b2c4-53fcbc62e585 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:54.658895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9177c-c08c-488b-b2c4-53fcbc62e585 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:54.663520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:54.664121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.664437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.664714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.665359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:57:54.665666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.667296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.679041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1101/2197] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:57:54.683304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fcdd-436c-4fe4-81bf-70bb015486ee None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:54.692913 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fcdd-436c-4fe4-81bf-70bb015486ee None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:54.697702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:54.698291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.698570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.698870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.699497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.701434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.728593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1097/2198] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:57:54 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:54.733248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:54.743220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f7544-d9a2-427a-8ac6-5d43077d1268 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:54.799132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:54.799697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.799978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.800285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.800917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.802890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.823067 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1102/2199] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:57:54 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:57:54.827739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:54.838281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7dd9-99c9-4167-9200-cf14e873d86b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:54.843516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:54.844113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.844427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.844703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.845346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.846987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.859741 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1098/2200] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:54.861242 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:54.871947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33a76f-cfc8-41fa-b725-3841ea13ab9b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:54.876666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae8a0e-6ef4-4dd8-a9cd-56cf3be7627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:54.877258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae8a0e-6ef4-4dd8-a9cd-56cf3be7627d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.877540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae8a0e-6ef4-4dd8-a9cd-56cf3be7627d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.877798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae8a0e-6ef4-4dd8-a9cd-56cf3be7627d admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.878439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae8a0e-6ef4-4dd8-a9cd-56cf3be7627d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.880006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae8a0e-6ef4-4dd8-a9cd-56cf3be7627d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.891266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1103/2201] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:57:54.899440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:54.910218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d922-cc92-4c30-bbd3-1584ac47bd76 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:54.914962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:54.915613 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.915896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.916234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] PATH_INFO: `/v3/projects/59727625ef364023997932413b8116f2/users/58e2fc893e8a4424a77e0ca17f2920c6/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.930630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59727625ef364023997932413b8116f2, user_id=58e2fc893e8a4424a77e0ca17f2920c6, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:54.933368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:54.946369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1099/2202] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:57:54 2022] PUT /identity/v3/projects/59727625ef364023997932413b8116f2/users/58e2fc893e8a4424a77e0ca17f2920c6/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:57:54.957352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67732e-fd22-4bc4-a65e-4dbae9c9a423 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:54.957821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67732e-fd22-4bc4-a65e-4dbae9c9a423 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:54.958170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67732e-fd22-4bc4-a65e-4dbae9c9a423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:54.986296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-dc67732e-fd22-4bc4-a65e-4dbae9c9a423 None None] MFA Rules not processed for user `58e2fc893e8a4424a77e0ca17f2920c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:54.987517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-dc67732e-fd22-4bc4-a65e-4dbae9c9a423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:55.051732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1104/2203] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:55.062539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:55.064218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:55.091095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:55.123481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62afed-2445-4076-b05e-59895e050be4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:55.127361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:55.127971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:55.128281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:55.128557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:55.137956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:55.139408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:55.148456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1100/2204] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:57:56.537414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158723da-6fc5-49ed-87fb-dcb5045f9b64 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:56.537987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158723da-6fc5-49ed-87fb-dcb5045f9b64 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:56.537987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158723da-6fc5-49ed-87fb-dcb5045f9b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:56.556680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-158723da-6fc5-49ed-87fb-dcb5045f9b64 None None] MFA Rules not processed for user `58e2fc893e8a4424a77e0ca17f2920c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:57:56.557920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-158723da-6fc5-49ed-87fb-dcb5045f9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:56.565195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1105/2205] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:57:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:57:57.496269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:57:57.497750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:57.630751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67028c-eccf-4ced-ae30-b22c66396074 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:57:57.638635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71deb80-d6e9-4d43-af5f-750ed889ca7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:57:57.640992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71deb80-d6e9-4d43-af5f-750ed889ca7e service placement] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:57:57.641332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71deb80-d6e9-4d43-af5f-750ed889ca7e service placement] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:57:57.641782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71deb80-d6e9-4d43-af5f-750ed889ca7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:57:57.644027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b71deb80-d6e9-4d43-af5f-750ed889ca7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:57:57.659074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71deb80-d6e9-4d43-af5f-750ed889ca7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:57:57.660750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71deb80-d6e9-4d43-af5f-750ed889ca7e service placement] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:57:57.672203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1101/2206] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:57:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:58:03.751752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939edda-2736-43cf-8544-86f307248118 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:03.752237 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939edda-2736-43cf-8544-86f307248118 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:03.752423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939edda-2736-43cf-8544-86f307248118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:03.771566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e939edda-2736-43cf-8544-86f307248118 None None] MFA Rules not processed for user `c7fac765624c4e0b85b01f356a14a1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:03.772929 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e939edda-2736-43cf-8544-86f307248118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:03.785334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:03.828086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1106/2207] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:07.666936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=48 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:11.235025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:11.235647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:11.235647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:11.255010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] MFA Rules not processed for user `5d1f341b9aff4b7db13e429b86ba145e`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:11.256254 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:11.321641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1102/2208] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:13.820963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=17 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:21.313285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=25 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:43.851293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de20914-65ee-4c7b-8e2c-472a8cc7dcff None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:43.851801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de20914-65ee-4c7b-8e2c-472a8cc7dcff None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:43.851903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de20914-65ee-4c7b-8e2c-472a8cc7dcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:43.871236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0de20914-65ee-4c7b-8e2c-472a8cc7dcff None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:43.872456 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0de20914-65ee-4c7b-8e2c-472a8cc7dcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:43.926565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1107/2209] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:58:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:43.970983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:43.972401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:44.043564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f46ac-f451-4ecb-b132-5d01cdc27589 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:44.049247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:44.049876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:44.050160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:44.050433 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:44.060199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:44.061781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:44.070362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1103/2210] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:58:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:58:44.574855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221df0-d54f-4c64-802b-41580a2b9726 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:44.575149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221df0-d54f-4c64-802b-41580a2b9726 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:44.575431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221df0-d54f-4c64-802b-41580a2b9726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:44.594830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e2221df0-d54f-4c64-802b-41580a2b9726 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:44.596039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e2221df0-d54f-4c64-802b-41580a2b9726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:44.603196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1108/2211] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 10 16:58:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:44.616082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:44.627213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39b8bb-d302-4313-bc6d-df29002ce2a1 service swift] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:44.632249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:44.632867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:44.633142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:44.633412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:44.641399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:44.642972 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:44.650858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1104/2212] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:58:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:58:44.878103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594565f-6bc0-409b-8b49-38d296b5f521 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:44.878558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594565f-6bc0-409b-8b49-38d296b5f521 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:44.878648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594565f-6bc0-409b-8b49-38d296b5f521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:44.897001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a594565f-6bc0-409b-8b49-38d296b5f521 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:44.898178 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a594565f-6bc0-409b-8b49-38d296b5f521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:44.905314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1109/2213] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:58:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:44.917513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:44.928265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be668469-8cf0-43a4-8f08-fde7b77eb73e service swift] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:44.933240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:44.933824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:44.934098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:44.934365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:44.942991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:44.944644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:44.952526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1105/2214] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:58:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:58:45.307991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecc6dd-134e-4830-a613-d34aa571edb4 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:45.308314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecc6dd-134e-4830-a613-d34aa571edb4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:45.308593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecc6dd-134e-4830-a613-d34aa571edb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:45.327939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f0ecc6dd-134e-4830-a613-d34aa571edb4 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:45.329099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f0ecc6dd-134e-4830-a613-d34aa571edb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:45.336416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1110/2215] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 10 16:58:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:45.349477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:45.360002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7813e-f1ff-45fb-97c5-a5c08c0e95f1 service swift] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:45.365222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284db22-fc14-4f58-a250-146d0d15cd3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:45.365815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284db22-fc14-4f58-a250-146d0d15cd3a service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:45.366097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284db22-fc14-4f58-a250-146d0d15cd3a service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:45.366370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284db22-fc14-4f58-a250-146d0d15cd3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:45.374472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284db22-fc14-4f58-a250-146d0d15cd3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:45.376038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284db22-fc14-4f58-a250-146d0d15cd3a service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:45.384000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1106/2216] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:58:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:58:48.517636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecc6dd-134e-4830-a613-d34aa571edb4 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:48.580796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecc6dd-134e-4830-a613-d34aa571edb4 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:48.585649 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:48.586287 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.586567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.586839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] PATH_INFO: `/v3/users/58e2fc893e8a4424a77e0ca17f2920c6` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.587962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e2fc893e8a4424a77e0ca17f2920c6)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:48.589906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:48.645842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e2fc893e8a4424a77e0ca17f2920c6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:48.647007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e2fc893e8a4424a77e0ca17f2920c6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:48.658419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e2fc893e8a4424a77e0ca17f2920c6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:48.659526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e2fc893e8a4424a77e0ca17f2920c6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:48.677131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e2fc893e8a4424a77e0ca17f2920c6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:48.693841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1111/2217] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:58:48 2022] DELETE /identity/v3/users/58e2fc893e8a4424a77e0ca17f2920c6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:58:48.698374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.698826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.698956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.721023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:48.722724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:48.813900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1107/2218] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:48.824056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:48.825528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:48.839065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:48.908203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120298e5-cbd2-4ae5-b25e-303b99829041 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:48.914029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb9bad-1af4-42ed-8bc4-c37f61528dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:48.914759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9bad-1af4-42ed-8bc4-c37f61528dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.915145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9bad-1af4-42ed-8bc4-c37f61528dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.915513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9bad-1af4-42ed-8bc4-c37f61528dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.925309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb9bad-1af4-42ed-8bc4-c37f61528dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:48.926757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb9bad-1af4-42ed-8bc4-c37f61528dd6 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:48.937632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1112/2219] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:58:49.187840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.189893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:49.202686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241024-c3b8-4c0b-8e1c-0bb7bb63aa32 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.207653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.208565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.208845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.209114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] PATH_INFO: `/v3/projects/59727625ef364023997932413b8116f2` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.210235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59727625ef364023997932413b8116f2)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.212156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.259358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:49.262759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59727625ef364023997932413b8116f2', 'request_id': 'req-390eef75-1790-4d85-8421-3ce21dbce144'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:49.268882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59727625ef364023997932413b8116f2', 'request_id': 'req-390eef75-1790-4d85-8421-3ce21dbce144'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:49.274898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59727625ef364023997932413b8116f2', 'request_id': 'req-390eef75-1790-4d85-8421-3ce21dbce144'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:49.287019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] The token cache is being invalidate because project 59727625ef364023997932413b8116f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:58:49.287250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:49.287875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-390eef75-1790-4d85-8421-3ce21dbce144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:58:49.289386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1108/2220] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:58:49 2022] DELETE /identity/v3/projects/59727625ef364023997932413b8116f2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:58:49.294353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869844fe-211d-4d1e-8f65-ab1184891714 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.294670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869844fe-211d-4d1e-8f65-ab1184891714 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.294938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869844fe-211d-4d1e-8f65-ab1184891714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.315107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-869844fe-211d-4d1e-8f65-ab1184891714 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:49.316358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-869844fe-211d-4d1e-8f65-ab1184891714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:49.378511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1113/2221] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:49.386697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdba3a-3030-44a9-a175-77a5ebf5c066 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.387005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdba3a-3030-44a9-a175-77a5ebf5c066 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.387282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdba3a-3030-44a9-a175-77a5ebf5c066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.407689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5fbdba3a-3030-44a9-a175-77a5ebf5c066 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:49.408960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5fbdba3a-3030-44a9-a175-77a5ebf5c066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:49.417120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1109/2222] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:49.421495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869844fe-211d-4d1e-8f65-ab1184891714 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.432231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869844fe-211d-4d1e-8f65-ab1184891714 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.436920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.437529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.437852 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.438120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.438762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:49.439063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.440678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.452342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1114/2223] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:49.456599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdba3a-3030-44a9-a175-77a5ebf5c066 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.466610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdba3a-3030-44a9-a175-77a5ebf5c066 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.471952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.472594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.472878 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.473154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.473778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.475726 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.506749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1110/2224] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:58:49 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:58:49.507945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.518093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac6b1f-4ff9-4a6a-9893-aaac0c49b2bc admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.581049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.581677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.581954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.582227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.582895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.585174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.616627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1115/2225] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:58:49 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:58:49.617409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.629995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48088f43-083c-4170-a16f-8c7cf7c5939f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.636489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.636489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.636489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.636489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.636983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.638580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.649720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1111/2226] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:58:49.654119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.664800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406279f-7e7e-4783-975e-2a2c3b422b4e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.669520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e88672-0fff-43f4-991a-62926089d63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.670121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e88672-0fff-43f4-991a-62926089d63d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.670403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e88672-0fff-43f4-991a-62926089d63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.670677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e88672-0fff-43f4-991a-62926089d63d admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.671320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e88672-0fff-43f4-991a-62926089d63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.673594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e88672-0fff-43f4-991a-62926089d63d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.684062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1116/2227] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:58:49.688615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.698449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5813-02e9-4a99-bcd5-86aa3b427ea2 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.703049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.703850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.704136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.704434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] PATH_INFO: `/v3/projects/41311d9aa3f54596a61033b0ea82ba80/users/a1c78f05a0f24151a1884faed65d286e/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.720846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41311d9aa3f54596a61033b0ea82ba80, user_id=a1c78f05a0f24151a1884faed65d286e, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.723591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.737779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1112/2228] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:58:49 2022] PUT /identity/v3/projects/41311d9aa3f54596a61033b0ea82ba80/users/a1c78f05a0f24151a1884faed65d286e/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:58:49.742727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda75b-1cb5-4a8d-87ac-83d19caa80c7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.743018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda75b-1cb5-4a8d-87ac-83d19caa80c7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.743293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda75b-1cb5-4a8d-87ac-83d19caa80c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.762745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9ccda75b-1cb5-4a8d-87ac-83d19caa80c7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:49.763955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9ccda75b-1cb5-4a8d-87ac-83d19caa80c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:49.772578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:49.828494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1117/2229] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:49.838411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.840095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:49.900014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b54a70-c93a-42a5-8f35-38cb3a59df64 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.903662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.904299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.904578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.904848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.914242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.915668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.925632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1113/2230] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:58:54.111220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37d53-9b44-4f34-818c-052a9f924837 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.111784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37d53-9b44-4f34-818c-052a9f924837 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.111882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37d53-9b44-4f34-818c-052a9f924837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.150721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f8a37d53-9b44-4f34-818c-052a9f924837 None None] MFA Rules not processed for user `a1c78f05a0f24151a1884faed65d286e`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:54.151975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f8a37d53-9b44-4f34-818c-052a9f924837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:54.253790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1118/2231] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:58:54.258289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:54.269505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d560f8-6633-49b8-bac5-eebedfd72bc8 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:54.274595 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:54.276653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.276921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.277253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.281321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:54.283356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:54.313655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1114/2232] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:58:54 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:58:54.317729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a37d53-9b44-4f34-818c-052a9f924837 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:54.328927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a37d53-9b44-4f34-818c-052a9f924837 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:54.388782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:54.389470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.389690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.389967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.390607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:54.392587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:54.416078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1119/2233] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:58:54 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:58:54.420429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:54.436236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede890bb-a2eb-4357-b7ae-825c822da72f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:54.442890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:54.443601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.443956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.444266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.444922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:54.446540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:54.458416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1115/2234] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:58:54.462594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:54.473383 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a2ca-b257-4bc8-81c7-b7d84c6cbcb8 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:54.478297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:54.478971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.479250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.479522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] PATH_INFO: `/v3/projects/b822d87e462e4e3d8f340858ca213ace/users/7bf5bb13c628487f8773438d7f3058e0/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.496133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b822d87e462e4e3d8f340858ca213ace, user_id=7bf5bb13c628487f8773438d7f3058e0, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:54.498825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:54.514751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1120/2235] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:58:54 2022] PUT /identity/v3/projects/b822d87e462e4e3d8f340858ca213ace/users/7bf5bb13c628487f8773438d7f3058e0/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:58:54.519047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:54.530300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab929b8-7370-49fd-8bfa-8c619b8919bd admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:54.603945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:54.631123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be328cc-79d2-4db5-bd6f-6aa5b6f9f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:54.632260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be328cc-79d2-4db5-bd6f-6aa5b6f9f2bb admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.632754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be328cc-79d2-4db5-bd6f-6aa5b6f9f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.633276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be328cc-79d2-4db5-bd6f-6aa5b6f9f2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.634174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be328cc-79d2-4db5-bd6f-6aa5b6f9f2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:54.636197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be328cc-79d2-4db5-bd6f-6aa5b6f9f2bb admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:54.647720 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1116/2236] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:58:54.652048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:54.663470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fad18b-2f45-4048-a6ab-703ea6beb498 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:54.668571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e74692-2f85-4f1e-9d01-dc8dd0497299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:54.669281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74692-2f85-4f1e-9d01-dc8dd0497299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.669568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74692-2f85-4f1e-9d01-dc8dd0497299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.669843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74692-2f85-4f1e-9d01-dc8dd0497299 admin admin] PATH_INFO: `/v3/projects/b822d87e462e4e3d8f340858ca213ace/users/7bf5bb13c628487f8773438d7f3058e0/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.672725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e74692-2f85-4f1e-9d01-dc8dd0497299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b822d87e462e4e3d8f340858ca213ace, user_id=7bf5bb13c628487f8773438d7f3058e0, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:54.675371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e74692-2f85-4f1e-9d01-dc8dd0497299 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:54.678797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:54.692592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1121/2237] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:58:54 2022] PUT /identity/v3/projects/b822d87e462e4e3d8f340858ca213ace/users/7bf5bb13c628487f8773438d7f3058e0/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:58:54.802752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40c98-8431-4a51-9597-31f63039548e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:54.804373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40c98-8431-4a51-9597-31f63039548e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:54.804590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40c98-8431-4a51-9597-31f63039548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:54.828172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ded40c98-8431-4a51-9597-31f63039548e None None] MFA Rules not processed for user `409b789be2d9402885b2498435c76286`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:54.829877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ded40c98-8431-4a51-9597-31f63039548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:54.940077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1117/2238] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:58:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:00.607043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec293b03-bb5e-4cb8-b064-e1e022c2cd7e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:00.607651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec293b03-bb5e-4cb8-b064-e1e022c2cd7e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:00.607651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec293b03-bb5e-4cb8-b064-e1e022c2cd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:00.643592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ec293b03-bb5e-4cb8-b064-e1e022c2cd7e None None] MFA Rules not processed for user `7bf5bb13c628487f8773438d7f3058e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:00.644857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ec293b03-bb5e-4cb8-b064-e1e022c2cd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:00.835807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1122/2239] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:59:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:00.837562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded40c98-8431-4a51-9597-31f63039548e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:00.839010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ded40c98-8431-4a51-9597-31f63039548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:00.972491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded40c98-8431-4a51-9597-31f63039548e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:00.980984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:00.981782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:00.982182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:00.982494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:01.070984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:01.082495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:01.104324 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1118/2240] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:59:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:59:10.804658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=36 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:11.097955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:12.630334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b70cf2-359c-46bc-8d5f-2dc705a2ed83 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:12.631029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b70cf2-359c-46bc-8d5f-2dc705a2ed83 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:12.631029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b70cf2-359c-46bc-8d5f-2dc705a2ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:12.651119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-32b70cf2-359c-46bc-8d5f-2dc705a2ed83 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:12.652401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-32b70cf2-359c-46bc-8d5f-2dc705a2ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:12.707990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1123/2241] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 16:59:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:12.720452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:12.721840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:12.810892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992302-2ea6-4099-abf4-5155dda80b8a service nova] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:12.818008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a9b3b-2c3e-4304-8a3e-f23e51c92ff6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:12.818630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a9b3b-2c3e-4304-8a3e-f23e51c92ff6 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:12.818912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a9b3b-2c3e-4304-8a3e-f23e51c92ff6 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:12.819187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a9b3b-2c3e-4304-8a3e-f23e51c92ff6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:12.829905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a9b3b-2c3e-4304-8a3e-f23e51c92ff6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:12.829905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a9b3b-2c3e-4304-8a3e-f23e51c92ff6 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:12.841017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1119/2242] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:59:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:59:22.700654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=25 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:22.835652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=32 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:31.405057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0a317-13e1-4f48-83b7-d86eb47b097d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:31.406258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0a317-13e1-4f48-83b7-d86eb47b097d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:31.406758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0a317-13e1-4f48-83b7-d86eb47b097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:31.430934 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f9c0a317-13e1-4f48-83b7-d86eb47b097d None None] MFA Rules not processed for user `7a701149974b45db9f782bc5a232aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:31.432334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f9c0a317-13e1-4f48-83b7-d86eb47b097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:31.499016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1124/2243] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:59:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:36.542786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:36.543240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:36.543374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:36.571724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:36.572975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:36.580978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1120/2244] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 10 16:59:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:36.596091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0a317-13e1-4f48-83b7-d86eb47b097d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:36.607854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0a317-13e1-4f48-83b7-d86eb47b097d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:36.612895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:36.613506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:36.613786 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:36.614058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:36.622261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:36.623843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:36.632434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1125/2245] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 16:59:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:59:38.926705 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:38.928326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:38.996448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0d75d-b37e-48c7-9d3c-8bd34d3a2930 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.001208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.001888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.002174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.002449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] PATH_INFO: `/v3/users/5d1f341b9aff4b7db13e429b86ba145e` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.003585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1f341b9aff4b7db13e429b86ba145e)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.005585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.052587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1f341b9aff4b7db13e429b86ba145e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.059756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1f341b9aff4b7db13e429b86ba145e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.067799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1f341b9aff4b7db13e429b86ba145e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.068600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1f341b9aff4b7db13e429b86ba145e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.081573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-022ce870-99fc-40f3-a861-8f166d5bd4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1f341b9aff4b7db13e429b86ba145e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.087653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1121/2246] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 16:59:38 2022] DELETE /identity/v3/users/5d1f341b9aff4b7db13e429b86ba145e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:59:39.259725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.273077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03934870-3dcf-4612-8d20-e47f78c27b07 service swift] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.340304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.340975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.341266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.341538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] PATH_INFO: `/v3/projects/abe3003a0c91471e9982327ac5e17b9e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.342685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe3003a0c91471e9982327ac5e17b9e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.344756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.393970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe3003a0c91471e9982327ac5e17b9e', 'request_id': 'req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.399412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe3003a0c91471e9982327ac5e17b9e', 'request_id': 'req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.405610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe3003a0c91471e9982327ac5e17b9e', 'request_id': 'req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.423621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] The token cache is being invalidate because project abe3003a0c91471e9982327ac5e17b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 16:59:39.424007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.424828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6e2c7574-c0f0-4623-8ec7-0e5fb8cca14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 16:59:39.429670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1126/2247] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 16:59:39 2022] DELETE /identity/v3/projects/abe3003a0c91471e9982327ac5e17b9e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:59:39.431351 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b0688-f779-461d-b806-97545b26065c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.431640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b0688-f779-461d-b806-97545b26065c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.431918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b0688-f779-461d-b806-97545b26065c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.452850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1e9b0688-f779-461d-b806-97545b26065c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:39.454068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1e9b0688-f779-461d-b806-97545b26065c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:39.517021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1122/2248] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:39.525052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec594ee-671e-44fc-ab13-53cd503b34d2 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.525403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec594ee-671e-44fc-ab13-53cd503b34d2 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.525672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec594ee-671e-44fc-ab13-53cd503b34d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.546433 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8ec594ee-671e-44fc-ab13-53cd503b34d2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:39.547637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8ec594ee-671e-44fc-ab13-53cd503b34d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:39.555535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1127/2249] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:39.559594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b0688-f779-461d-b806-97545b26065c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.569784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b0688-f779-461d-b806-97545b26065c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.574377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.574968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.575248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.575521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.576230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:59:39.576534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.578176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.587482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:39.591428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1123/2250] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 16:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:59:39.595452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec594ee-671e-44fc-ab13-53cd503b34d2 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.604777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec594ee-671e-44fc-ab13-53cd503b34d2 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.609495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.610091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.610371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.610642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.611264 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.613198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.627963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:39.642053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1128/2251] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 16:59:39 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:59:39.646631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.656887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6023-f0eb-43be-8a09-2862cd3ba673 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.713368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.713968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.714244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.714515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.715165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.717133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.738129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1124/2252] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 16:59:39 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:59:39.742444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.753062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda78c6-9c99-413a-9064-34ce2ff1037d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.757646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.758240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.758517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.758789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.759431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.761054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.771379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1129/2253] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:59:39.775245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.785138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ac22e-9275-40d5-9350-7af413e135c1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.789893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e4581-cd2b-4a24-b315-950fc3b2bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.790524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e4581-cd2b-4a24-b315-950fc3b2bed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.790807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e4581-cd2b-4a24-b315-950fc3b2bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.791094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e4581-cd2b-4a24-b315-950fc3b2bed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.791731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e4581-cd2b-4a24-b315-950fc3b2bed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.793410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e4581-cd2b-4a24-b315-950fc3b2bed2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.804053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1125/2254] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 16:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:59:39.808758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.822276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa2dd8-f578-4702-a350-bb4fc1b090dc admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:39.829244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:39.830453 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.830741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.831008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] PATH_INFO: `/v3/projects/acb2377a7b484a269d2d9f97e7f58f91/users/a67d0e49a7cb49f38ab5fe4c4e27ed78/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.851065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb2377a7b484a269d2d9f97e7f58f91, user_id=a67d0e49a7cb49f38ab5fe4c4e27ed78, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:39.853817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:39.868752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1130/2255] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 16:59:39 2022] PUT /identity/v3/projects/acb2377a7b484a269d2d9f97e7f58f91/users/a67d0e49a7cb49f38ab5fe4c4e27ed78/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:59:39.880703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:39.880993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:39.881267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:39.912568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] MFA Rules not processed for user `a67d0e49a7cb49f38ab5fe4c4e27ed78`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:39.913988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:39.984000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1126/2256] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:59:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:39.996136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:39.997881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:40.055417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c887c-45d7-4ff3-af4b-0d75f5f9fa74 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:40.059192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c76a4a9747469d88a0a1a5748452a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:40.059795 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:40.060282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:40.060601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:40.069509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:40.070922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:40.079527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1131/2257] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:59:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:59:45.150497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:45.151885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:45.211772 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a6e55-5e21-4c66-b141-2bf0b51a6b97 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:45.215674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17e219-2793-462d-8219-53db87faab19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:45.216359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17e219-2793-462d-8219-53db87faab19 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:45.216642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17e219-2793-462d-8219-53db87faab19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:45.217013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17e219-2793-462d-8219-53db87faab19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:45.219100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-eb17e219-2793-462d-8219-53db87faab19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:45.240985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:45.305378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:45.307399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:45.319553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17e219-2793-462d-8219-53db87faab19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:45.321329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17e219-2793-462d-8219-53db87faab19 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:45.324136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa849b5a-0255-416e-a571-d7c46f11601e service glance] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:45.329087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:45.330302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:45.330713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:45.331073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:45.332903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:45.339859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1127/2258] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:59:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:59:45.349672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:45.351381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:45.363181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1132/2259] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:59:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 16:59:45.882635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:45.882924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:45.883203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:45.902668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] MFA Rules not processed for user `a1c78f05a0f24151a1884faed65d286e`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:59:45.903873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:45.964717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1128/2260] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 16:59:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 16:59:55.358032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:INSERT=2,keystone:SELECT=49 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:55.957707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=48 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:57.019987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:59:57.022168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:57.036800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9195bc-0655-46c2-aa26-7eb9df38e239 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:59:57.043246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1075d4b-bc21-4f7e-8124-3e66326f4a6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:59:57.043853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1075d4b-bc21-4f7e-8124-3e66326f4a6a service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:59:57.044131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1075d4b-bc21-4f7e-8124-3e66326f4a6a service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:59:57.044437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1075d4b-bc21-4f7e-8124-3e66326f4a6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:59:57.046349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d1075d4b-bc21-4f7e-8124-3e66326f4a6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:59:57.113134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1075d4b-bc21-4f7e-8124-3e66326f4a6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:59:57.115186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1075d4b-bc21-4f7e-8124-3e66326f4a6a service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:59:57.123935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1133/2261] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 16:59:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:00:02.057975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:02.059452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:02.076529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa8eb8-65be-4376-8e53-f93fc6b5437a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:02.081939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:02.082538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:02.083438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:02.083964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:02.085663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:02.142395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:02.144026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:02.153625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1129/2262] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:07.120031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=28 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:00:12.149388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=28 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:00:13.973920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c496-0817-419e-8326-17c44f253edb None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:13.974449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c496-0817-419e-8326-17c44f253edb None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:13.974449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c496-0817-419e-8326-17c44f253edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:13.995008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6f20c496-0817-419e-8326-17c44f253edb None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:00:13.996379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6f20c496-0817-419e-8326-17c44f253edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:14.054744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1134/2263] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 17:00:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:00:14.077791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:14.079271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:14.151779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4fa14-1115-4ee0-8e0f-d0fa3c15ea84 service nova] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:14.156999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:14.157629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:14.158030 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:14.158316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:14.169495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:14.171112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:14.182089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1130/2264] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:00:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:00:16.939490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c496-0817-419e-8326-17c44f253edb None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:16.957290 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c496-0817-419e-8326-17c44f253edb None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:16.965168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:16.965797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:16.966079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:16.966358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:16.968110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:17.040438 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:17.042429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:17.055388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1135/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:24.182748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=32 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:00:24.942651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:24.951997 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e529a3a-2953-4c2a-9a41-bfa630ae0dd5 service swift] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:24.955528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:24.956167 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:24.956466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:24.956736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:24.959241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:24.968657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:24.970070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:24.978570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1131/2266] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:27.048139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=53 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:00:34.423084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:34.434888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c8655-3309-4c3a-a27d-9cdb975898fd service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:34.440653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:34.441836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:34.442121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:34.442389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:34.444101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:34.457760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:34.460169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:34.471294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1136/2267] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:34.975127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:00:35.260393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:35.269094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47a60b-122a-462e-be8d-dc4deaee2653 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:35.272812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:35.273411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:35.273693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:35.273965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:35.275685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:35.339694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:35.341164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:35.350962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1132/2268] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:37.105811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:37.122875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba614bfc-0ea9-42cb-8752-55c318e72e05 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:37.130860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b118dd1-26b3-491d-8681-6308e13f6dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:37.131555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b118dd1-26b3-491d-8681-6308e13f6dad service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:37.131840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b118dd1-26b3-491d-8681-6308e13f6dad service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:37.132113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b118dd1-26b3-491d-8681-6308e13f6dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:37.134065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6b118dd1-26b3-491d-8681-6308e13f6dad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:37.147940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b118dd1-26b3-491d-8681-6308e13f6dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:37.149673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b118dd1-26b3-491d-8681-6308e13f6dad service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:37.188229 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1137/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:39.025737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:39.034134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd882c5-2f35-4343-b5cb-da34426373f7 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:39.037742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed57dd-b8f7-4f63-a49f-33634b126196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:39.038331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed57dd-b8f7-4f63-a49f-33634b126196 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.038605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed57dd-b8f7-4f63-a49f-33634b126196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.038873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed57dd-b8f7-4f63-a49f-33634b126196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.040589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-59ed57dd-b8f7-4f63-a49f-33634b126196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:39.049801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed57dd-b8f7-4f63-a49f-33634b126196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:39.051209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed57dd-b8f7-4f63-a49f-33634b126196 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:39.059172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1133/2270] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:39.183514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae2df-1417-42b2-bf2b-242db5f395ed None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.184083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae2df-1417-42b2-bf2b-242db5f395ed None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.184083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae2df-1417-42b2-bf2b-242db5f395ed None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.185568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1138/2271] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 17:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:00:39.192691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992556-0d57-4dd5-8786-0452c655c360 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.192990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992556-0d57-4dd5-8786-0452c655c360 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.193260 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992556-0d57-4dd5-8786-0452c655c360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.214382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-15992556-0d57-4dd5-8786-0452c655c360 None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:00:39.215618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-15992556-0d57-4dd5-8786-0452c655c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:39.225012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1134/2272] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 17:00:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:00:39.236309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ae2df-1417-42b2-bf2b-242db5f395ed None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:39.248312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ae2df-1417-42b2-bf2b-242db5f395ed None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:39.251798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6a467-2ea6-4c05-a662-6aa6808fe342 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c76a4a9747469d88a0a1a5748452a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:39.252461 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6a467-2ea6-4c05-a662-6aa6808fe342 service glance] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.252702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6a467-2ea6-4c05-a662-6aa6808fe342 service glance] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.253128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6a467-2ea6-4c05-a662-6aa6808fe342 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.264316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6a467-2ea6-4c05-a662-6aa6808fe342 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:39.265732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6a467-2ea6-4c05-a662-6aa6808fe342 service glance] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:39.277844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1139/2273] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:39.350323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016b2d-71e8-42b5-a623-bde3f328b8f2 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.350751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016b2d-71e8-42b5-a623-bde3f328b8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.351095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016b2d-71e8-42b5-a623-bde3f328b8f2 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.352676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1135/2274] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 17:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:00:39.357601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16e859-8972-48a9-9437-174c5803312b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.357892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16e859-8972-48a9-9437-174c5803312b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.358166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16e859-8972-48a9-9437-174c5803312b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.380354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9c16e859-8972-48a9-9437-174c5803312b None None] MFA Rules not processed for user `256af8436dd842fdb4ebe90ec9be26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:00:39.381585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9c16e859-8972-48a9-9437-174c5803312b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:39.388717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1140/2275] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 17:00:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:00:39.400529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b016b2d-71e8-42b5-a623-bde3f328b8f2 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:39.411591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b016b2d-71e8-42b5-a623-bde3f328b8f2 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:39.415395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9b963-c639-4431-b47b-8d3e99c83b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:39.416105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9b963-c639-4431-b47b-8d3e99c83b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.416486 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9b963-c639-4431-b47b-8d3e99c83b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.416844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9b963-c639-4431-b47b-8d3e99c83b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.425020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9b963-c639-4431-b47b-8d3e99c83b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:39.427198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9b963-c639-4431-b47b-8d3e99c83b22 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:39.435886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1136/2276] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:39.518011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f61e16-0495-4e33-9024-ba9233a87d79 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.518302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f61e16-0495-4e33-9024-ba9233a87d79 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.518579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f61e16-0495-4e33-9024-ba9233a87d79 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.520016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1141/2277] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 17:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:00:39.525201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61561592-6763-4094-be63-f77ec4223e46 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.525489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61561592-6763-4094-be63-f77ec4223e46 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.525793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61561592-6763-4094-be63-f77ec4223e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.545149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-61561592-6763-4094-be63-f77ec4223e46 None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:00:39.546290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-61561592-6763-4094-be63-f77ec4223e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:39.556125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1137/2278] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 17:00:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:00:39.566947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f61e16-0495-4e33-9024-ba9233a87d79 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:39.577803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f61e16-0495-4e33-9024-ba9233a87d79 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:39.581495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05fa21-10b4-44dd-a0b8-a36a022a6823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:39.582120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05fa21-10b4-44dd-a0b8-a36a022a6823 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.582400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05fa21-10b4-44dd-a0b8-a36a022a6823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.582671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05fa21-10b4-44dd-a0b8-a36a022a6823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.592989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05fa21-10b4-44dd-a0b8-a36a022a6823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:39.594411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05fa21-10b4-44dd-a0b8-a36a022a6823 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:39.605803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1142/2279] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:39.670227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61561592-6763-4094-be63-f77ec4223e46 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:39.671661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-61561592-6763-4094-be63-f77ec4223e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:39.742420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61561592-6763-4094-be63-f77ec4223e46 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:39.747496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:39.748101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:39.748408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:39.748677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:39.750398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:39.761968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:39.763532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:39.773586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1138/2280] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:49.601277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=27 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:00:49.769670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=80 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:00:54.718346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204403c-030a-4021-9fa2-3689cb72a889 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:54.718828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204403c-030a-4021-9fa2-3689cb72a889 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:54.719092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204403c-030a-4021-9fa2-3689cb72a889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:54.740065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4204403c-030a-4021-9fa2-3689cb72a889 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:00:54.741432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4204403c-030a-4021-9fa2-3689cb72a889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:54.748760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1143/2281] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 17:00:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:00:54.761810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:54.773408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bcec4-058c-40cf-a51a-90b62b427806 service placement] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:54.778798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af52328-ccd3-414c-9ef1-e60aeedd5bb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:54.779413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52328-ccd3-414c-9ef1-e60aeedd5bb8 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:54.779691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52328-ccd3-414c-9ef1-e60aeedd5bb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:54.779962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52328-ccd3-414c-9ef1-e60aeedd5bb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:54.789695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af52328-ccd3-414c-9ef1-e60aeedd5bb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:54.791285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af52328-ccd3-414c-9ef1-e60aeedd5bb8 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:54.801027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1139/2282] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:00:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:00:57.538380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204403c-030a-4021-9fa2-3689cb72a889 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:57.622069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204403c-030a-4021-9fa2-3689cb72a889 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:57.626809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:57.627468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:57.627816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:57.628045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] PATH_INFO: `/v3/users/a67d0e49a7cb49f38ab5fe4c4e27ed78` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:57.629595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67d0e49a7cb49f38ab5fe4c4e27ed78)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:57.631564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:57.694532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67d0e49a7cb49f38ab5fe4c4e27ed78'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:57.695272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67d0e49a7cb49f38ab5fe4c4e27ed78'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:57.703539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67d0e49a7cb49f38ab5fe4c4e27ed78'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:57.704479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67d0e49a7cb49f38ab5fe4c4e27ed78'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:57.723452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67d0e49a7cb49f38ab5fe4c4e27ed78'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:57.739847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1144/2283] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:00:57 2022] DELETE /identity/v3/users/a67d0e49a7cb49f38ab5fe4c4e27ed78 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:00:57.744539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:57.745083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:57.745292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:57.765037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:00:57.766576 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:57.835871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1140/2284] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:00:57.845204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:57.846653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:57.908038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e023c77-9e65-4da2-93e5-15ff03ac657a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:57.912357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:57.912977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:57.913259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:57.913530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:57.923096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:57.924575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:57.935399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1145/2285] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:00:58.240224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:00:58.241655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:00:58.255581 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcabe3-3dc1-49b7-88c4-58dde6175592 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:00:58.272979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:00:58.274124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:00:58.274560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:00:58.274971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] PATH_INFO: `/v3/projects/acb2377a7b484a269d2d9f97e7f58f91` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:00:58.276406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb2377a7b484a269d2d9f97e7f58f91)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:00:58.280241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:00:58.349017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb2377a7b484a269d2d9f97e7f58f91', 'request_id': 'req-3e45d5d7-66c6-486f-ba50-cf48302af81f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:58.355127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb2377a7b484a269d2d9f97e7f58f91', 'request_id': 'req-3e45d5d7-66c6-486f-ba50-cf48302af81f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:58.362507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb2377a7b484a269d2d9f97e7f58f91', 'request_id': 'req-3e45d5d7-66c6-486f-ba50-cf48302af81f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:58.374877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] The token cache is being invalidate because project acb2377a7b484a269d2d9f97e7f58f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:00:58.375165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:58.375776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:00:58.377302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1141/2286] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:00:58 2022] DELETE /identity/v3/projects/acb2377a7b484a269d2d9f97e7f58f91 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:01:06.296011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:06.297593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:06.314713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b566-670b-45ea-947f-d94e866f6500 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:06.327751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:06.375088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:06.375726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:06.376006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:06.376333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] PATH_INFO: `/v3/users/409b789be2d9402885b2498435c76286` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:06.377563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409b789be2d9402885b2498435c76286)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:06.380119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:06.440844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409b789be2d9402885b2498435c76286'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.442374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409b789be2d9402885b2498435c76286'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.451126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409b789be2d9402885b2498435c76286'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.451971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409b789be2d9402885b2498435c76286'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.464598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-bcb47470-9365-4b65-a8c6-c2ace9c5c6c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409b789be2d9402885b2498435c76286'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.477664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1146/2287] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:01:06 2022] DELETE /identity/v3/users/409b789be2d9402885b2498435c76286 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:01:06.683239 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:06.685445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:06.714212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d5d7-66c6-486f-ba50-cf48302af81f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:06.802950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:06.803948 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:06.804413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:06.804862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] PATH_INFO: `/v3/projects/aa1a38abca5848b78518b882982fc1be` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:06.806271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1a38abca5848b78518b882982fc1be)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:06.808244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:06.823441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:06.872272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1a38abca5848b78518b882982fc1be', 'request_id': 'req-239dead6-bcd9-4c72-b28c-f6864293247a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.881991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1a38abca5848b78518b882982fc1be', 'request_id': 'req-239dead6-bcd9-4c72-b28c-f6864293247a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.888690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1a38abca5848b78518b882982fc1be', 'request_id': 'req-239dead6-bcd9-4c72-b28c-f6864293247a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.903328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] The token cache is being invalidate because project aa1a38abca5848b78518b882982fc1be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:01:06.903620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.904347 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-239dead6-bcd9-4c72-b28c-f6864293247a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:01:06.905867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1142/2288] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:01:06 2022] DELETE /identity/v3/projects/aa1a38abca5848b78518b882982fc1be => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:01:06.911678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7cf82-96f0-4fa1-aacc-023073ced31a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:06.911974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7cf82-96f0-4fa1-aacc-023073ced31a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:06.912293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7cf82-96f0-4fa1-aacc-023073ced31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:06.933207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a8a7cf82-96f0-4fa1-aacc-023073ced31a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:06.934758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a8a7cf82-96f0-4fa1-aacc-023073ced31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:07.006093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1147/2289] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:01:07.007899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd154d-989c-41e1-8c86-694db3365aa4 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.008215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd154d-989c-41e1-8c86-694db3365aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.008494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd154d-989c-41e1-8c86-694db3365aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.029150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-28fd154d-989c-41e1-8c86-694db3365aa4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:07.030397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-28fd154d-989c-41e1-8c86-694db3365aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:07.039075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1143/2290] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:01:07.043295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7cf82-96f0-4fa1-aacc-023073ced31a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:07.053374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7cf82-96f0-4fa1-aacc-023073ced31a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:07.058377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:07.058992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.059269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.059538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.060217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:01:07.060522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:07.062162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:07.076643 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1148/2291] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:01:07.080814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd154d-989c-41e1-8c86-694db3365aa4 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:07.091123 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd154d-989c-41e1-8c86-694db3365aa4 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:07.096133 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:07.096785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.097066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.097338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.098593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:07.100557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:07.135664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1144/2292] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:01:07 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:01:07.139545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:07.151808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2654d9-8e3a-4706-a3ac-f7c2544fd1f1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:07.172928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:07.215092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:07.215706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.215983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.216293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.216924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:07.218896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:07.244112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1149/2293] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:01:07 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:01:07.245494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:07.256643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff133e2-11a7-4b4e-98a2-de8617ba900c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:07.261626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:07.262375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.262774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.263129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.263882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:07.265609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:07.276989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1145/2294] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:01:07.281190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:07.292877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ab3c-dd17-412e-9bc3-6e968238e85c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:07.298056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8cdaa-34bc-4735-8f59-287510f32510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:07.298726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8cdaa-34bc-4735-8f59-287510f32510 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.299032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8cdaa-34bc-4735-8f59-287510f32510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.299305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8cdaa-34bc-4735-8f59-287510f32510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.299949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8cdaa-34bc-4735-8f59-287510f32510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:07.301620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8cdaa-34bc-4735-8f59-287510f32510 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:07.316120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1150/2295] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:01:07.317920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:07.328582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e14ccb-eeb5-4357-9957-9be80e2d4c10 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:07.336909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:07.337619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.337907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.338190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] PATH_INFO: `/v3/projects/4c868216664144beb0a3a0360c56e637/users/5d0f55fa4717472db61860fd31f5e427/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.355045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c868216664144beb0a3a0360c56e637, user_id=5d0f55fa4717472db61860fd31f5e427, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:07.358261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:07.374376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1146/2296] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:01:07 2022] PUT /identity/v3/projects/4c868216664144beb0a3a0360c56e637/users/5d0f55fa4717472db61860fd31f5e427/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:01:07.380051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f47a8-153d-4ee0-b66e-41ad6c912cb4 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.380713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f47a8-153d-4ee0-b66e-41ad6c912cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.381096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f47a8-153d-4ee0-b66e-41ad6c912cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.402420 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-374f47a8-153d-4ee0-b66e-41ad6c912cb4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:07.403861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-374f47a8-153d-4ee0-b66e-41ad6c912cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:07.474107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1151/2297] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:01:07.483655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:07.485440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:07.545502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbe27e-38ea-4a17-9ddf-c2793f1a8a5f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:07.549176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:07.549786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:07.550064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:07.550336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:07.560343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:07.561774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:07.571980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1147/2298] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:01:11.697341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da221665-2890-407c-9e43-a2d6f7e34c35 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:11.697924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da221665-2890-407c-9e43-a2d6f7e34c35 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:11.697924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da221665-2890-407c-9e43-a2d6f7e34c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:11.732798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-da221665-2890-407c-9e43-a2d6f7e34c35 None None] MFA Rules not processed for user `5d0f55fa4717472db61860fd31f5e427`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:11.734086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-da221665-2890-407c-9e43-a2d6f7e34c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:11.810758 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1152/2299] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:01:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:01:11.815396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:11.826027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa33394-b803-41dc-90cb-fc3a7b458348 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:11.830769 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:11.831379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:11.831664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:11.832640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:11.833280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:11.835478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:11.865862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1148/2300] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:01:11 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:01:11.869928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da221665-2890-407c-9e43-a2d6f7e34c35 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:11.881656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da221665-2890-407c-9e43-a2d6f7e34c35 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:11.911269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:11.942707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:11.943384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:11.943670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:11.943943 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:11.944605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:11.946694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:11.979701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1153/2301] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:01:11 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:01:11.983293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:11.995313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c329d-3b4d-4976-9845-78988003bc29 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:12.000237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:12.000826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:12.001106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:12.001380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:12.002044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:12.004119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:12.015682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1149/2302] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:01:12.021273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:12.032504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41a62f-9bbe-48f6-ba64-a2a85cf2cc5f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:12.037431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:12.038107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:12.038387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:12.038661 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] PATH_INFO: `/v3/projects/acb03e7a0a8e4db4b42cd3ddccdd0ad8/users/1000044760e545b2b5e35ef49a339953/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:12.054526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb03e7a0a8e4db4b42cd3ddccdd0ad8, user_id=1000044760e545b2b5e35ef49a339953, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:12.057225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:12.072889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1154/2303] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:01:12 2022] PUT /identity/v3/projects/acb03e7a0a8e4db4b42cd3ddccdd0ad8/users/1000044760e545b2b5e35ef49a339953/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:01:12.077442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:12.088612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadf7f9-33bc-4b41-ad56-37f5a788be83 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:12.093425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:12.162004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a3c85-e3ae-4569-b914-3ed52f6331d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:12.162620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a3c85-e3ae-4569-b914-3ed52f6331d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:12.162893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a3c85-e3ae-4569-b914-3ed52f6331d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:12.163159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a3c85-e3ae-4569-b914-3ed52f6331d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:12.163812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a3c85-e3ae-4569-b914-3ed52f6331d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:12.165597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a3c85-e3ae-4569-b914-3ed52f6331d4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:12.177164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1150/2304] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:01:12.181405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:12.193249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8081b-ab7a-42bd-bbe6-fd8d4a0f59f0 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:12.198080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:12.198828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:12.199179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:12.199539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] PATH_INFO: `/v3/projects/acb03e7a0a8e4db4b42cd3ddccdd0ad8/users/1000044760e545b2b5e35ef49a339953/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:12.201590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb03e7a0a8e4db4b42cd3ddccdd0ad8, user_id=1000044760e545b2b5e35ef49a339953, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:12.204384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:12.216656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1155/2305] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:01:12 2022] PUT /identity/v3/projects/acb03e7a0a8e4db4b42cd3ddccdd0ad8/users/1000044760e545b2b5e35ef49a339953/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:01:17.092639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b9eec-25cf-400f-a790-238ab8e96667 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:17.093174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b9eec-25cf-400f-a790-238ab8e96667 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:17.093431 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b9eec-25cf-400f-a790-238ab8e96667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:17.127477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-fa3b9eec-25cf-400f-a790-238ab8e96667 None None] MFA Rules not processed for user `1000044760e545b2b5e35ef49a339953`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:17.129218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fa3b9eec-25cf-400f-a790-238ab8e96667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:17.221949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1151/2306] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:01:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:01:17.237048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:17.239546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:17.334808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35209-b9d0-4505-adb3-dc49ee2b2c30 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:17.346409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71c2f-5657-45b2-aaee-f09fe3655016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:17.347031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71c2f-5657-45b2-aaee-f09fe3655016 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:17.347359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71c2f-5657-45b2-aaee-f09fe3655016 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:17.348228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71c2f-5657-45b2-aaee-f09fe3655016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:17.423847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71c2f-5657-45b2-aaee-f09fe3655016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:17.426183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71c2f-5657-45b2-aaee-f09fe3655016 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:17.437007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1156/2307] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:01:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:01:18.492080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:18.492670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:18.492794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:18.512297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] MFA Rules not processed for user `5d0f55fa4717472db61860fd31f5e427`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:18.513628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:18.521496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1152/2308] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:01:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:01:27.357983 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555ea4-408d-477c-a26b-21abe6f65752 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:27.358347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555ea4-408d-477c-a26b-21abe6f65752 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:27.358483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555ea4-408d-477c-a26b-21abe6f65752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:27.381899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-70555ea4-408d-477c-a26b-21abe6f65752 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:27.383046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-70555ea4-408d-477c-a26b-21abe6f65752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:27.397967 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:27.444620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1157/2309] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 10 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:01:27.466530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:27.468022 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:27.538411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424b808-dac6-41ac-b363-f1724e4fa5a0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:27.544020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:27.544724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:27.545006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:27.545280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:27.554419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:27.556014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:27.564017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1153/2310] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:01:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:01:31.659511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555ea4-408d-477c-a26b-21abe6f65752 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:31.673781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555ea4-408d-477c-a26b-21abe6f65752 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:31.679166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:31.679762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:31.680038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:31.680348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:31.682039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:31.739980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:31.741613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:31.750731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1158/2311] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:01:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:01:37.560749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:41.746504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=44 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:45.641815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:45.714092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c71d3-bf00-4ffa-b7b0-744362551ab3 service swift] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:45.717660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0cf2b-bd40-4e32-9809-388c271550e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:45.718257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0cf2b-bd40-4e32-9809-388c271550e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:45.718534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0cf2b-bd40-4e32-9809-388c271550e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:45.718805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0cf2b-bd40-4e32-9809-388c271550e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:45.720604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3ee0cf2b-bd40-4e32-9809-388c271550e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:45.730368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0cf2b-bd40-4e32-9809-388c271550e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:45.731788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0cf2b-bd40-4e32-9809-388c271550e8 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:45.739466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1154/2312] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:01:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:01:48.994193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:01:49.006913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9a8b-0670-45f9-bbbf-bd4c0a7bdf62 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:01:49.025933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:01:49.026754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:49.027094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:49.027449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:49.034120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:49.053385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:01:49.055204 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:01:49.069393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1159/2313] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:01:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:01:55.736096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=26 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:01:57.480482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5b1a-9c98-4536-8381-4a86c4dc36ec None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:01:57.481093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5b1a-9c98-4536-8381-4a86c4dc36ec None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:01:57.481093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5b1a-9c98-4536-8381-4a86c4dc36ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:01:57.499717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f53d5b1a-9c98-4536-8381-4a86c4dc36ec None None] MFA Rules not processed for user `c7fac765624c4e0b85b01f356a14a1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:01:57.500937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f53d5b1a-9c98-4536-8381-4a86c4dc36ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:01:57.507730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1155/2314] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:01:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:01:59.065662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:02:07.494491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:02:09.157065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:02:09.166872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28cf8e-f92b-43e9-aeb5-6a34962daf75 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:02:09.170440 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3f99e-e5fd-4120-8e53-9faa1e96e0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:02:09.171034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3f99e-e5fd-4120-8e53-9faa1e96e0ec service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:02:09.171313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3f99e-e5fd-4120-8e53-9faa1e96e0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:02:09.171587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3f99e-e5fd-4120-8e53-9faa1e96e0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:02:09.180337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3f99e-e5fd-4120-8e53-9faa1e96e0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:02:09.181788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3f99e-e5fd-4120-8e53-9faa1e96e0ec service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:02:09.191027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1160/2315] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:02:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:02:19.186883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:02:38.807569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:02:38.808103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:02:38.808103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:02:38.826425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] MFA Rules not processed for user `7a701149974b45db9f782bc5a232aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:02:38.827607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:02:38.881514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1156/2316] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:02:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:02:48.874983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=25 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:02:59.940325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b46899-68bb-4a2e-9dd7-754fc1583a32 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:02:59.940905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b46899-68bb-4a2e-9dd7-754fc1583a32 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:02:59.940905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b46899-68bb-4a2e-9dd7-754fc1583a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:02:59.969791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-76b46899-68bb-4a2e-9dd7-754fc1583a32 None None] MFA Rules not processed for user `96495ad2ad764ba3a0fb1e3bc28fd8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:02:59.970972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-76b46899-68bb-4a2e-9dd7-754fc1583a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:00.040271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1161/2317] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:02:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:01.113029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:01.114477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:01.253294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074d83d-d43f-47ff-884e-83cf3fe20cf3 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:01.258325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:01.258928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:01.259205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:01.259476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:01.261219 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:01.272371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:01.273951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:01.283782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1157/2318] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:10.038132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=37 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:11.279669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=63 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:34.741500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:34.741995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:34.742102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:34.761922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] MFA Rules not processed for user `a1c78f05a0f24151a1884faed65d286e`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:34.763115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:34.817294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1162/2319] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:43.382661 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:43.384047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:43.418147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a708c4-3b15-475a-9c3d-cb7c9386227f service placement] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:43.421919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:43.422559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:43.422841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:43.423117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:43.425256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:43.508933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:43.510274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:43.529230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1158/2320] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:44.809770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=25 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:46.545134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:46.548735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:46.583260 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df9687-6b39-4754-8350-8c0dabf05473 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:46.589213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:46.590717 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:46.591375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:46.591814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:46.594436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:46.618969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:46.620744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:46.645592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1163/2321] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:46.649002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:46.650739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:46.666028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e5397-69f1-465c-a037-b9917ee4aa6c service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:46.671086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:46.671727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:46.672223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:46.672556 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] PATH_INFO: `/v3/users/c7fac765624c4e0b85b01f356a14a1cb` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:46.674114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7fac765624c4e0b85b01f356a14a1cb)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:46.676609 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:46.727238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7fac765624c4e0b85b01f356a14a1cb'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:46.741153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7fac765624c4e0b85b01f356a14a1cb'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:46.750132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7fac765624c4e0b85b01f356a14a1cb'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:46.750790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7fac765624c4e0b85b01f356a14a1cb'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:46.763645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fbf1eef5-7fba-4bfd-92b9-37b8d76a112b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7fac765624c4e0b85b01f356a14a1cb'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:46.772665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1159/2322] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:03:46 2022] DELETE /identity/v3/users/c7fac765624c4e0b85b01f356a14a1cb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:46.950678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:46.962928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ce744-7ee5-4cf3-bf0d-dbcb4be1f413 service placement] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.032963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.033505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.033784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.034059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] PATH_INFO: `/v3/projects/36cf7bbfa5d3420996a2af036cf8e0da` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.035169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cf7bbfa5d3420996a2af036cf8e0da)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.037145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.083049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cf7bbfa5d3420996a2af036cf8e0da', 'request_id': 'req-cde5c936-81de-4d69-b1f9-aacfb9afec88'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:47.088988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cf7bbfa5d3420996a2af036cf8e0da', 'request_id': 'req-cde5c936-81de-4d69-b1f9-aacfb9afec88'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:47.095035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cf7bbfa5d3420996a2af036cf8e0da', 'request_id': 'req-cde5c936-81de-4d69-b1f9-aacfb9afec88'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:47.107063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] The token cache is being invalidate because project 36cf7bbfa5d3420996a2af036cf8e0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:03:47.107353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:47.107972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-cde5c936-81de-4d69-b1f9-aacfb9afec88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:47.109488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1164/2323] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:03:46 2022] DELETE /identity/v3/projects/36cf7bbfa5d3420996a2af036cf8e0da => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:47.114419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a0d9a-0dc0-4a65-9df5-1c0b642655b7 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.114710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a0d9a-0dc0-4a65-9df5-1c0b642655b7 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.114993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a0d9a-0dc0-4a65-9df5-1c0b642655b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.138014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:47.174715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ed9a0d9a-0dc0-4a65-9df5-1c0b642655b7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:47.175939 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ed9a0d9a-0dc0-4a65-9df5-1c0b642655b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:47.236377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1160/2324] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:47.244051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4752e-f683-47bb-8f92-a1b48936b1e8 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.244359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4752e-f683-47bb-8f92-a1b48936b1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.244820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4752e-f683-47bb-8f92-a1b48936b1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.267486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5df4752e-f683-47bb-8f92-a1b48936b1e8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:47.268886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5df4752e-f683-47bb-8f92-a1b48936b1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:47.277667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1165/2325] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:03:47.282026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a0d9a-0dc0-4a65-9df5-1c0b642655b7 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:47.291719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a0d9a-0dc0-4a65-9df5-1c0b642655b7 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.296298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.296930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.297200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.297472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.298115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:03:47.298423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.300054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.312810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1161/2326] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:03:47.317685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4752e-f683-47bb-8f92-a1b48936b1e8 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:47.327419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4752e-f683-47bb-8f92-a1b48936b1e8 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.332922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.333579 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.333810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.334084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.335157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.337121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.364505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1166/2327] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:03:47 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:47.368607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:47.378526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58abd1-67f7-4673-ba79-217ed894e4ae admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.435131 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.435746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.436027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.436344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.436996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.438922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.461122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1162/2328] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:03:47 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:03:47.465246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:47.476358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78a6d6-cd7d-418c-8611-1f019fd4c460 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.480933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.481524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.481879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.482128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.483088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.484706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.494602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1167/2329] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:47.498261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:47.508002 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc99b0-c0bf-413d-ad62-2a66f65b1739 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.512527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e678164-29a9-420b-9b04-2b38d0d71b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.513113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e678164-29a9-420b-9b04-2b38d0d71b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.513389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e678164-29a9-420b-9b04-2b38d0d71b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.513658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e678164-29a9-420b-9b04-2b38d0d71b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.514291 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e678164-29a9-420b-9b04-2b38d0d71b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.515866 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e678164-29a9-420b-9b04-2b38d0d71b95 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.526678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1163/2330] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:47.530583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:47.541311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b298a8-a7c1-412e-bacf-32d1812c65b6 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.545926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.546573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.546858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.547131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] PATH_INFO: `/v3/projects/045ddbc4de514d4b93c57d9c4bf9fa1a/users/3f76da26f5f74b82b7c12c0da990f0c7/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.561235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=045ddbc4de514d4b93c57d9c4bf9fa1a, user_id=3f76da26f5f74b82b7c12c0da990f0c7, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.563873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.576834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1168/2331] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:47 2022] PUT /identity/v3/projects/045ddbc4de514d4b93c57d9c4bf9fa1a/users/3f76da26f5f74b82b7c12c0da990f0c7/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:47.588534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4510c-a117-4e07-bd90-be6ec6df7b3c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.589007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4510c-a117-4e07-bd90-be6ec6df7b3c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.589285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4510c-a117-4e07-bd90-be6ec6df7b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.618799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f8b4510c-a117-4e07-bd90-be6ec6df7b3c None None] MFA Rules not processed for user `3f76da26f5f74b82b7c12c0da990f0c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:47.619996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f8b4510c-a117-4e07-bd90-be6ec6df7b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:47.662952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:47.692496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1164/2332] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:47.703775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:47.705515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:47.724540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:47.817377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56c41e-3060-47e1-9163-2a5532720045 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:47.824780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c610e-85e6-489f-b39e-9fdcb4d8126a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:47.825400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c610e-85e6-489f-b39e-9fdcb4d8126a service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:47.825677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c610e-85e6-489f-b39e-9fdcb4d8126a service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:47.826580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c610e-85e6-489f-b39e-9fdcb4d8126a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:47.836131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c610e-85e6-489f-b39e-9fdcb4d8126a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:47.837955 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c610e-85e6-489f-b39e-9fdcb4d8126a service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:47.847397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1169/2333] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:50.712518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4510c-a117-4e07-bd90-be6ec6df7b3c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:50.818131 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4510c-a117-4e07-bd90-be6ec6df7b3c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:50.839392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:50.840068 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:50.840388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:50.840693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] PATH_INFO: `/v3/users/3f76da26f5f74b82b7c12c0da990f0c7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:50.841989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f76da26f5f74b82b7c12c0da990f0c7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:50.843829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:50.899897 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f76da26f5f74b82b7c12c0da990f0c7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:50.908468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f76da26f5f74b82b7c12c0da990f0c7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:50.921780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f76da26f5f74b82b7c12c0da990f0c7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:50.922756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f76da26f5f74b82b7c12c0da990f0c7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:50.936041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f76da26f5f74b82b7c12c0da990f0c7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:50.944171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1165/2334] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:03:50 2022] DELETE /identity/v3/users/3f76da26f5f74b82b7c12c0da990f0c7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:50.948450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:50.949457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:50.949839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:50.972799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:50.974126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:51.041236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1170/2335] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:51.051514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:51.053037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:51.117264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6768ab-53d0-4ce9-ad12-185f1e82aa0f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:51.121150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df5dfd-1780-4ca7-b10d-3f22af5c05ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:51.121770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5dfd-1780-4ca7-b10d-3f22af5c05ab service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.122048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5dfd-1780-4ca7-b10d-3f22af5c05ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.122350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5dfd-1780-4ca7-b10d-3f22af5c05ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.133682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df5dfd-1780-4ca7-b10d-3f22af5c05ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:51.135266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df5dfd-1780-4ca7-b10d-3f22af5c05ab service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:51.146656 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1166/2336] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:51.464905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:51.466775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:51.501701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60750279-cdc0-41a9-97d0-972760b1c665 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:51.509870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:51.510539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.510875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.511155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] PATH_INFO: `/v3/projects/045ddbc4de514d4b93c57d9c4bf9fa1a` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.512389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=045ddbc4de514d4b93c57d9c4bf9fa1a)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:51.514533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:51.576607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045ddbc4de514d4b93c57d9c4bf9fa1a', 'request_id': 'req-ff84e4f6-f559-4029-8530-26528eca3804'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:51.583687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045ddbc4de514d4b93c57d9c4bf9fa1a', 'request_id': 'req-ff84e4f6-f559-4029-8530-26528eca3804'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:51.592416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045ddbc4de514d4b93c57d9c4bf9fa1a', 'request_id': 'req-ff84e4f6-f559-4029-8530-26528eca3804'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:51.597194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:51.616009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] The token cache is being invalidate because project 045ddbc4de514d4b93c57d9c4bf9fa1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:03:51.616374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:51.617148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff84e4f6-f559-4029-8530-26528eca3804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:51.618725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1171/2337] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:03:51 2022] DELETE /identity/v3/projects/045ddbc4de514d4b93c57d9c4bf9fa1a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:51.624523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c43e1-638d-43c5-aa72-f4d9558b3ae6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.624875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c43e1-638d-43c5-aa72-f4d9558b3ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.625100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c43e1-638d-43c5-aa72-f4d9558b3ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.637460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:51.651819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f88c43e1-638d-43c5-aa72-f4d9558b3ae6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:51.653841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f88c43e1-638d-43c5-aa72-f4d9558b3ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:51.726951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1167/2338] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:51.728628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56203e-db53-4db0-82f8-d6a7d79ddd58 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.728927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56203e-db53-4db0-82f8-d6a7d79ddd58 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.729263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56203e-db53-4db0-82f8-d6a7d79ddd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.754070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-cb56203e-db53-4db0-82f8-d6a7d79ddd58 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:51.755730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-cb56203e-db53-4db0-82f8-d6a7d79ddd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:51.763905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1172/2339] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:03:51.768518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c43e1-638d-43c5-aa72-f4d9558b3ae6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:51.780971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c43e1-638d-43c5-aa72-f4d9558b3ae6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:51.790306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:51.790959 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.791242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.791517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.792251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:03:51.792564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:51.794169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:51.807765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1168/2340] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:03:51.812063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56203e-db53-4db0-82f8-d6a7d79ddd58 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:51.821732 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56203e-db53-4db0-82f8-d6a7d79ddd58 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:51.827037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:51.827669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.827946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.828252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.828877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:51.830827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:51.870855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1173/2341] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:03:51 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:03:51.871994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:51.881986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd906f9-dddc-4401-bff7-dc5540e1ea07 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:51.942666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:51.945796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.946107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.946391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.947045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:51.949116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:51.972676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1169/2342] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:03:51 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:51.977578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:51.989314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70a4b6-acec-4048-bf81-0597d707ef1f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:51.994545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:51.995231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:51.995582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:51.995922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:51.996656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:51.998263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.136547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1174/2343] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:52.140592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.151164 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe32441-957a-4a23-9c60-0d926339bc57 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.156038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fc262-d83f-49aa-9989-ef7a89000f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.156967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fc262-d83f-49aa-9989-ef7a89000f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.156967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fc262-d83f-49aa-9989-ef7a89000f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.157235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fc262-d83f-49aa-9989-ef7a89000f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.157932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fc262-d83f-49aa-9989-ef7a89000f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.159912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fc262-d83f-49aa-9989-ef7a89000f13 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.172412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1170/2344] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:03:52.176517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.186487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75ffd1-0b00-4866-92ea-700bf84abf19 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.191132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.192315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.192599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.192925 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] PATH_INFO: `/v3/projects/3109bbe1b6fa45eba9ed6fe35918e3fc/users/53c3641f795c4b5b9be020736b6bf3dc/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.208521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3109bbe1b6fa45eba9ed6fe35918e3fc, user_id=53c3641f795c4b5b9be020736b6bf3dc, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.211459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.226098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1175/2345] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:52 2022] PUT /identity/v3/projects/3109bbe1b6fa45eba9ed6fe35918e3fc/users/53c3641f795c4b5b9be020736b6bf3dc/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:03:52.236397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19546159-fed5-4f22-8ae0-f2fe84e682c6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.236748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19546159-fed5-4f22-8ae0-f2fe84e682c6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.237206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19546159-fed5-4f22-8ae0-f2fe84e682c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.267308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-19546159-fed5-4f22-8ae0-f2fe84e682c6 None None] MFA Rules not processed for user `53c3641f795c4b5b9be020736b6bf3dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:52.268732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-19546159-fed5-4f22-8ae0-f2fe84e682c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:52.340978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:52.345875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1171/2346] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:52.351251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.417380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584f5a9-bce1-455b-8d54-69a810462faa admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.422224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.422829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.423105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.423378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.424018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.425956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.458360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1176/2347] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:03:52 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:52.460038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19546159-fed5-4f22-8ae0-f2fe84e682c6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.472204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19546159-fed5-4f22-8ae0-f2fe84e682c6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.536413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.537152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.537507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.537843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.538574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.540599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.570553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1172/2348] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:03:52 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:52.575404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.585900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb9b3e-ea4f-40ab-a072-1ff4a02608c8 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.593299 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.594159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.594441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.594728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.596221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.598246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.614112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1177/2349] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:52.615478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.626684 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ad185-e1df-477c-9c9a-c706f815dac4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.632502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92b8d-ffd3-4ed9-8657-16e8cb753dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.633277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92b8d-ffd3-4ed9-8657-16e8cb753dee admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.633653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92b8d-ffd3-4ed9-8657-16e8cb753dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.634024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92b8d-ffd3-4ed9-8657-16e8cb753dee admin admin] PATH_INFO: `/v3/projects/ac65c2f963ff4e7d88beebdf8e529bda/users/de73bac8a64945be8748b4c406357d51/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.649854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb92b8d-ffd3-4ed9-8657-16e8cb753dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac65c2f963ff4e7d88beebdf8e529bda, user_id=de73bac8a64945be8748b4c406357d51, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.652762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb92b8d-ffd3-4ed9-8657-16e8cb753dee admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.665958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1173/2350] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:52 2022] PUT /identity/v3/projects/ac65c2f963ff4e7d88beebdf8e529bda/users/de73bac8a64945be8748b4c406357d51/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:52.670169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.683498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657659d8-c27c-44e1-be40-c6d92929c7b3 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.696863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e349f5-3143-431b-88eb-c4623ec40659 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.697174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e349f5-3143-431b-88eb-c4623ec40659 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.697514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e349f5-3143-431b-88eb-c4623ec40659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.716738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-76e349f5-3143-431b-88eb-c4623ec40659 None None] MFA Rules not processed for user `77b8847d10744ec98e0ed7a3f1e991ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:52.717851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-76e349f5-3143-431b-88eb-c4623ec40659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:52.741724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.742337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.742620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.743008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.743659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.745278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.755663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1178/2351] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:52.759816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.764697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:52.771371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9f7c4-dd2a-4ebc-ac5b-5611639fb76a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:52.777226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1174/2352] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:52.777699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:52.778202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.778523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.778802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] PATH_INFO: `/v3/projects/ac65c2f963ff4e7d88beebdf8e529bda/users/de73bac8a64945be8748b4c406357d51/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.780790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac65c2f963ff4e7d88beebdf8e529bda, user_id=de73bac8a64945be8748b4c406357d51, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:52.783504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:52.795164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1179/2353] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:52 2022] PUT /identity/v3/projects/ac65c2f963ff4e7d88beebdf8e529bda/users/de73bac8a64945be8748b4c406357d51/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:03:52.807549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:52.807925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:52.808304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:52.836794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] MFA Rules not processed for user `de73bac8a64945be8748b4c406357d51`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:52.838004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:52.911458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:52.924104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1175/2354] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:52.935573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:52.937403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:53.017626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1108275-6563-4a40-b8ee-3d2b97095870 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:53.022214 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:53.022928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:53.023090 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:53.023360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:53.033518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:53.034945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:53.044862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1180/2355] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:53.094910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:53.096355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:53.108383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e21360-98b4-43d1-a8c7-df9fa581f4c0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:53.112139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ce394-cdb8-4e6d-8128-b18d0ae9c2e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:53.112770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ce394-cdb8-4e6d-8128-b18d0ae9c2e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:53.113053 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ce394-cdb8-4e6d-8128-b18d0ae9c2e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:53.113329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ce394-cdb8-4e6d-8128-b18d0ae9c2e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:53.171147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ce394-cdb8-4e6d-8128-b18d0ae9c2e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:53.172685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ce394-cdb8-4e6d-8128-b18d0ae9c2e6 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:53.181810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1176/2356] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:56.566890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:56.627993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb32cb-c470-4f2d-90f9-a439a0df4300 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:56.632515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:56.633145 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:56.633428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:56.633702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] PATH_INFO: `/v3/users/53c3641f795c4b5b9be020736b6bf3dc` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:56.634829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c3641f795c4b5b9be020736b6bf3dc)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:56.636780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:56.681122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c3641f795c4b5b9be020736b6bf3dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:56.681773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c3641f795c4b5b9be020736b6bf3dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:56.688547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c3641f795c4b5b9be020736b6bf3dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:56.689252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c3641f795c4b5b9be020736b6bf3dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:56.701622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c3641f795c4b5b9be020736b6bf3dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:56.703515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:56.717111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1181/2357] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:03:56 2022] DELETE /identity/v3/users/53c3641f795c4b5b9be020736b6bf3dc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:56.721691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:56.722194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:56.722301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:56.740956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:56.742162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:56.802154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1177/2358] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:56.812003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:56.813519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:56.873277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363bb5-79f1-40c8-8f5e-168c2f91630f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:56.877108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:56.877724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:56.878056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:56.878277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:56.888157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:56.889600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:56.899722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1182/2359] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:57.161378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:57.162951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:57.179652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba30f5b-9693-40a2-a4a2-3d0fe1381290 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:57.185093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:57.185733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:57.186016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:57.186293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] PATH_INFO: `/v3/projects/3109bbe1b6fa45eba9ed6fe35918e3fc` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:57.187418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3109bbe1b6fa45eba9ed6fe35918e3fc)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:57.189388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:57.237339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3109bbe1b6fa45eba9ed6fe35918e3fc', 'request_id': 'req-0e881a4b-162a-485e-b350-ff7fabc01e17'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.243238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3109bbe1b6fa45eba9ed6fe35918e3fc', 'request_id': 'req-0e881a4b-162a-485e-b350-ff7fabc01e17'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.249450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3109bbe1b6fa45eba9ed6fe35918e3fc', 'request_id': 'req-0e881a4b-162a-485e-b350-ff7fabc01e17'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.262391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] The token cache is being invalidate because project 3109bbe1b6fa45eba9ed6fe35918e3fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:03:57.262676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.263303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.264809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1178/2360] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:03:57 2022] DELETE /identity/v3/projects/3109bbe1b6fa45eba9ed6fe35918e3fc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:57.268948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:57.279061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed33f36-7f74-401f-b8ee-a7b02200c93d service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:57.335888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:57.336573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:57.336850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:57.337116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] PATH_INFO: `/v3/users/de73bac8a64945be8748b4c406357d51` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:57.338241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de73bac8a64945be8748b4c406357d51)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:57.340138 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:57.384893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de73bac8a64945be8748b4c406357d51'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.385543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de73bac8a64945be8748b4c406357d51'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.392592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de73bac8a64945be8748b4c406357d51'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.393302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de73bac8a64945be8748b4c406357d51'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.405840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d4437865-1cd6-4953-ba5e-deee8a25d2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de73bac8a64945be8748b4c406357d51'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.418711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1183/2361] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:03:57 2022] DELETE /identity/v3/users/de73bac8a64945be8748b4c406357d51 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:57.685084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:57.686686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:57.701976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e881a4b-162a-485e-b350-ff7fabc01e17 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:57.715630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:57.762074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:57.762714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:57.762969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:57.763268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] PATH_INFO: `/v3/projects/ac65c2f963ff4e7d88beebdf8e529bda` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:57.764397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac65c2f963ff4e7d88beebdf8e529bda)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:57.766315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:57.814875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac65c2f963ff4e7d88beebdf8e529bda', 'request_id': 'req-5fe407c1-9795-426c-a470-8d0ed5915745'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.820379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac65c2f963ff4e7d88beebdf8e529bda', 'request_id': 'req-5fe407c1-9795-426c-a470-8d0ed5915745'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.826459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac65c2f963ff4e7d88beebdf8e529bda', 'request_id': 'req-5fe407c1-9795-426c-a470-8d0ed5915745'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.839708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] The token cache is being invalidate because project ac65c2f963ff4e7d88beebdf8e529bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:03:57.840028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.840862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5fe407c1-9795-426c-a470-8d0ed5915745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:03:57.842350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1179/2362] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:03:57 2022] DELETE /identity/v3/projects/ac65c2f963ff4e7d88beebdf8e529bda => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:57.847140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491fea1-7474-4b5f-a67d-5a7cbf86b55e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:57.848275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491fea1-7474-4b5f-a67d-5a7cbf86b55e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:57.848386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491fea1-7474-4b5f-a67d-5a7cbf86b55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:57.866992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c491fea1-7474-4b5f-a67d-5a7cbf86b55e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:57.868211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c491fea1-7474-4b5f-a67d-5a7cbf86b55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:57.876398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:57.929241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1184/2363] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:57.937042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df698ed-63e1-4237-a3b3-6a64ae1acaf8 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:57.937328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df698ed-63e1-4237-a3b3-6a64ae1acaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:57.937603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df698ed-63e1-4237-a3b3-6a64ae1acaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:57.957308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7df698ed-63e1-4237-a3b3-6a64ae1acaf8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:57.958506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7df698ed-63e1-4237-a3b3-6a64ae1acaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:57.966571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1180/2364] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:57.970739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491fea1-7474-4b5f-a67d-5a7cbf86b55e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:57.981008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491fea1-7474-4b5f-a67d-5a7cbf86b55e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:57.985608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:57.986202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:57.986480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:57.986783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:57.987419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:03:57.989195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:57.989321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.001849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1185/2365] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:03:58.006280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df698ed-63e1-4237-a3b3-6a64ae1acaf8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.016174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df698ed-63e1-4237-a3b3-6a64ae1acaf8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.021631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.022251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.022533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.022807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.023447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.025551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.053530 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1181/2366] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:58.057626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.067799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d75c-256b-4123-add3-deb4f4558ae1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.128390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.129728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.130006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.130273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.130937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.132879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.156091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1186/2367] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:58.166055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.176944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3b274-65bc-4c95-a837-7d0b5590f9e0 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.181742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.182308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.182585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.182860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.183497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.185128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.194994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1182/2368] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:58.198999 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.209089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8def-d4d3-4558-8fc1-98456a3072e7 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.213974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73d119-2b8e-402f-9bbf-e7ca5342232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.214566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d119-2b8e-402f-9bbf-e7ca5342232d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.214845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d119-2b8e-402f-9bbf-e7ca5342232d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.215115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d119-2b8e-402f-9bbf-e7ca5342232d admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.215783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73d119-2b8e-402f-9bbf-e7ca5342232d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.217429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73d119-2b8e-402f-9bbf-e7ca5342232d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.228566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1187/2369] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:58.232786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.243604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04090c80-8f3d-4c17-9866-e960a4c85d48 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.249271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.249919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.250197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.250469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] PATH_INFO: `/v3/projects/a5e918e82f864a35a6d33cba531a83e4/users/d16e371d7239469dafd3511faf6239f0/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.265949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e918e82f864a35a6d33cba531a83e4, user_id=d16e371d7239469dafd3511faf6239f0, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.268671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.281361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1183/2370] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:58 2022] PUT /identity/v3/projects/a5e918e82f864a35a6d33cba531a83e4/users/d16e371d7239469dafd3511faf6239f0/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:03:58.293444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3551-4301-4ef8-827a-94ad78decbf9 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.293734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3551-4301-4ef8-827a-94ad78decbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.294006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3551-4301-4ef8-827a-94ad78decbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.322305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-52fd3551-4301-4ef8-827a-94ad78decbf9 None None] MFA Rules not processed for user `d16e371d7239469dafd3511faf6239f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:58.323515 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-52fd3551-4301-4ef8-827a-94ad78decbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:58.386783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1188/2371] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:58.391381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.440509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:58.463865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e41a2-3dcb-4bb6-b3f4-6992dd033bb2 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.468676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.469288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.469571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.469847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.470478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.472415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.500313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1184/2372] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:58.505037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd3551-4301-4ef8-827a-94ad78decbf9 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.515635 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd3551-4301-4ef8-827a-94ad78decbf9 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.521348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:58.573656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.574261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.574541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.574814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.575437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.577381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.600381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1189/2373] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:03:58.604339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.616308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79cdc8-ad01-4fc1-8c97-426d1634d224 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.621340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.621943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.622222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.622493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.623162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.624787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.637152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1185/2374] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:58.641157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.651318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79db2e7-9599-40b5-bea6-0534fdb6cf1e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.656041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5f535-d9f7-4705-8dd3-2275183e1f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.656670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5f535-d9f7-4705-8dd3-2275183e1f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.656948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5f535-d9f7-4705-8dd3-2275183e1f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.657238 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5f535-d9f7-4705-8dd3-2275183e1f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.657878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5f535-d9f7-4705-8dd3-2275183e1f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.659449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5f535-d9f7-4705-8dd3-2275183e1f3a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.669476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1190/2375] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:03:58.673608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.684161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e6290-ee53-49f1-a441-17607c57c2bd admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.689222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.689821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.690105 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.690382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] PATH_INFO: `/v3/projects/6ec5203d4e5f4405b97c47cd9a965c06/users/cc0d51ea67404741b4a6bb345954258e/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.706534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec5203d4e5f4405b97c47cd9a965c06, user_id=cc0d51ea67404741b4a6bb345954258e, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.709264 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.723480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1186/2376] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:58 2022] PUT /identity/v3/projects/6ec5203d4e5f4405b97c47cd9a965c06/users/cc0d51ea67404741b4a6bb345954258e/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:58.733836 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf331-3b19-4fbc-87c1-abf8745aef23 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.734131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf331-3b19-4fbc-87c1-abf8745aef23 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.734407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf331-3b19-4fbc-87c1-abf8745aef23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.767028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-7edbf331-3b19-4fbc-87c1-abf8745aef23 None None] MFA Rules not processed for user `cc0d51ea67404741b4a6bb345954258e`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:58.768351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7edbf331-3b19-4fbc-87c1-abf8745aef23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:58.837099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1191/2377] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:03:58.841657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.905410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbbdfe-ca1e-4902-b3a1-ca0c19c16bf4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:58.910562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:58.911170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:58.911451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:58.911729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:58.912394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:58.914303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:58.942468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1187/2378] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:03:58.946331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbf331-3b19-4fbc-87c1-abf8745aef23 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:58.956903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbf331-3b19-4fbc-87c1-abf8745aef23 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:59.012740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:59.013353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.013627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.013898 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.014526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:59.016454 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:59.040018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1192/2379] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:03:58 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:03:59.043965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:59.055544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b65a7-2536-442b-a429-8c05c045b0bf admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:59.060467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:59.061071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.061349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.061618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.062263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:59.063858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:59.074775 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1188/2380] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:03:59.078963 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:59.092452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e805ce9-1fa5-4e9e-afcd-4a4ca2ea6eec admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:59.097295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:59.098473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.098844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.099141 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] PATH_INFO: `/v3/projects/c55efb68ff314533bf03a9bf844a2c88/users/c930d209e17d4110b6eeb919831f1b3c/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.101470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:59.118973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55efb68ff314533bf03a9bf844a2c88, user_id=c930d209e17d4110b6eeb919831f1b3c, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:59.121650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:59.136756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1193/2381] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:59 2022] PUT /identity/v3/projects/c55efb68ff314533bf03a9bf844a2c88/users/c930d209e17d4110b6eeb919831f1b3c/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:59.140453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:59.151249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cca1fd-9b65-4f3d-abb7-50511d3e21dd admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:59.216944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15255d7c-5909-49d3-b438-734d9457452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:59.217265 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15255d7c-5909-49d3-b438-734d9457452a admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.217620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15255d7c-5909-49d3-b438-734d9457452a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.217954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15255d7c-5909-49d3-b438-734d9457452a admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.218870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15255d7c-5909-49d3-b438-734d9457452a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:59.220521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15255d7c-5909-49d3-b438-734d9457452a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:59.232828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1189/2382] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:03:59.237212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:59.248156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592590a-dc1b-4e63-8e57-8fb3f8b69819 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:59.252990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:59.253683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.253966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.254243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] PATH_INFO: `/v3/projects/c55efb68ff314533bf03a9bf844a2c88/users/c930d209e17d4110b6eeb919831f1b3c/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.256208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55efb68ff314533bf03a9bf844a2c88, user_id=c930d209e17d4110b6eeb919831f1b3c, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:59.259561 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:59.275466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1194/2383] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:03:59 2022] PUT /identity/v3/projects/c55efb68ff314533bf03a9bf844a2c88/users/c930d209e17d4110b6eeb919831f1b3c/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:03:59.286307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.286680 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.286987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.295259 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:03:59.322529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] MFA Rules not processed for user `c930d209e17d4110b6eeb919831f1b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:03:59.324158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:59.397630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1190/2384] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:03:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:03:59.409078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:59.411329 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:59.468408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c7eeb-3580-49b1-8e4f-5b1b76467213 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:59.471950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:59.473189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.473469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.473740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.482576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:59.483991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:59.493333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1195/2385] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:03:59.534157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:03:59.536117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:03:59.547800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5012d-cba5-4f43-bbe3-e78671123d76 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:03:59.551459 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae8364-649f-44bb-bd72-9d82348ddfcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:03:59.552055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae8364-649f-44bb-bd72-9d82348ddfcb service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:03:59.552363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae8364-649f-44bb-bd72-9d82348ddfcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:03:59.552636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae8364-649f-44bb-bd72-9d82348ddfcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:03:59.608977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae8364-649f-44bb-bd72-9d82348ddfcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:03:59.610438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae8364-649f-44bb-bd72-9d82348ddfcb service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:03:59.624547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1191/2386] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:03:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:02.496891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:02.564070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f9fcb-5880-4b43-a03d-fff25c3c8672 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:02.568706 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:02.569360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:02.569637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:02.569908 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] PATH_INFO: `/v3/users/d16e371d7239469dafd3511faf6239f0` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:02.571284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] RBAC: Authorizing `identity:delete_user(user_id=d16e371d7239469dafd3511faf6239f0)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:02.573225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:02.622387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd16e371d7239469dafd3511faf6239f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:02.622960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd16e371d7239469dafd3511faf6239f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:02.630747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd16e371d7239469dafd3511faf6239f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:02.631702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd16e371d7239469dafd3511faf6239f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:02.644279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd16e371d7239469dafd3511faf6239f0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:02.646261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:02.660259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1196/2387] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:04:02 2022] DELETE /identity/v3/users/d16e371d7239469dafd3511faf6239f0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:02.665076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:02.665586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:02.665669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:02.685042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:02.686249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:02.754007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1192/2388] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:02.770134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:02.772249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:02.836319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250eeedd-4f29-4ebc-8222-4c9ed5f2cdab admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:02.840489 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:02.841103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:02.841386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:02.841656 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:02.851833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:02.853417 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:02.863844 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1197/2389] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:03.128406 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:03.129789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:03.141742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b552292-49c9-4511-a157-56d92ea9a863 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:03.146896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:03.147539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:03.147822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:03.148098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] PATH_INFO: `/v3/projects/a5e918e82f864a35a6d33cba531a83e4` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:03.149280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e918e82f864a35a6d33cba531a83e4)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:03.151193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:03.176523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:03.199687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e918e82f864a35a6d33cba531a83e4', 'request_id': 'req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.205280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e918e82f864a35a6d33cba531a83e4', 'request_id': 'req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.212220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e918e82f864a35a6d33cba531a83e4', 'request_id': 'req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.225870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] The token cache is being invalidate because project a5e918e82f864a35a6d33cba531a83e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:04:03.226155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.226888 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.228602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1193/2390] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:04:03 2022] DELETE /identity/v3/projects/a5e918e82f864a35a6d33cba531a83e4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:03.233034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:03.242773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86489f52-20ba-442d-b3a7-76f08146958e service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:03.297835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:03.298460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:03.298739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:03.299099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] PATH_INFO: `/v3/users/cc0d51ea67404741b4a6bb345954258e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:03.300245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0d51ea67404741b4a6bb345954258e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:03.302150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:03.347021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0d51ea67404741b4a6bb345954258e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.348211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0d51ea67404741b4a6bb345954258e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.355481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0d51ea67404741b4a6bb345954258e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.356096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0d51ea67404741b4a6bb345954258e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.368333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0d51ea67404741b4a6bb345954258e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.380773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1198/2391] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:04:03 2022] DELETE /identity/v3/users/cc0d51ea67404741b4a6bb345954258e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:03.651678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:03.655667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:03.672868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88ffbc-6a5d-40b3-92e0-60e25484a74f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:03.729517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:03.730142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:03.730448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:03.730718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] PATH_INFO: `/v3/projects/6ec5203d4e5f4405b97c47cd9a965c06` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:03.731820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec5203d4e5f4405b97c47cd9a965c06)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:03.733867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:03.779153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec5203d4e5f4405b97c47cd9a965c06', 'request_id': 'req-9d5b5788-0137-4fd8-8b06-5a14d7c66693'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.785585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec5203d4e5f4405b97c47cd9a965c06', 'request_id': 'req-9d5b5788-0137-4fd8-8b06-5a14d7c66693'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.791934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec5203d4e5f4405b97c47cd9a965c06', 'request_id': 'req-9d5b5788-0137-4fd8-8b06-5a14d7c66693'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.807596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] The token cache is being invalidate because project 6ec5203d4e5f4405b97c47cd9a965c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:04:03.808231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.808859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.810371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1194/2392] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:04:03 2022] DELETE /identity/v3/projects/6ec5203d4e5f4405b97c47cd9a965c06 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:04:03.814435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:03.815850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:03.828672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f65cb-cda1-47d0-b328-ed07f560833e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:03.838064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:03.887523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:03.888226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:03.888531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:03.888800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] PATH_INFO: `/v3/users/c930d209e17d4110b6eeb919831f1b3c` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:03.889923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c930d209e17d4110b6eeb919831f1b3c)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:03.891834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:03.943108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c930d209e17d4110b6eeb919831f1b3c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.943802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c930d209e17d4110b6eeb919831f1b3c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.952201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c930d209e17d4110b6eeb919831f1b3c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.952855 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c930d209e17d4110b6eeb919831f1b3c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.967781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-4accaf88-bc3c-4691-8b55-666367214c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c930d209e17d4110b6eeb919831f1b3c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:03.980789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1199/2393] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:04:03 2022] DELETE /identity/v3/users/c930d209e17d4110b6eeb919831f1b3c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:04.279845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:04.281296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:04.295202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5788-0137-4fd8-8b06-5a14d7c66693 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:04.341792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:04.360806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:04.361455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.361727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.362008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] PATH_INFO: `/v3/projects/c55efb68ff314533bf03a9bf844a2c88` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.363786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55efb68ff314533bf03a9bf844a2c88)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:04.365733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:04.411750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55efb68ff314533bf03a9bf844a2c88', 'request_id': 'req-7edef66b-ef87-41fd-b062-110df9028cf2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:04.417306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55efb68ff314533bf03a9bf844a2c88', 'request_id': 'req-7edef66b-ef87-41fd-b062-110df9028cf2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:04.423378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55efb68ff314533bf03a9bf844a2c88', 'request_id': 'req-7edef66b-ef87-41fd-b062-110df9028cf2'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:04.437410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] The token cache is being invalidate because project c55efb68ff314533bf03a9bf844a2c88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:04:04.437703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:04.438324 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-7edef66b-ef87-41fd-b062-110df9028cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:04.439870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1195/2394] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:04:04 2022] DELETE /identity/v3/projects/c55efb68ff314533bf03a9bf844a2c88 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:04.445940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3beba-562a-4da7-b28c-b715b2392948 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.446228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3beba-562a-4da7-b28c-b715b2392948 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.446504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3beba-562a-4da7-b28c-b715b2392948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.465774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a5c3beba-562a-4da7-b28c-b715b2392948 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:04.466960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a5c3beba-562a-4da7-b28c-b715b2392948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:04.524893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1200/2395] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:04:04.535514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe07dc-c804-406a-ae28-9b87dd7501d6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.535805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe07dc-c804-406a-ae28-9b87dd7501d6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.536084 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe07dc-c804-406a-ae28-9b87dd7501d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.556227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8dbe07dc-c804-406a-ae28-9b87dd7501d6 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:04.557482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8dbe07dc-c804-406a-ae28-9b87dd7501d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:04.565607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1196/2396] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:04.569942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3beba-562a-4da7-b28c-b715b2392948 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:04.579597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3beba-562a-4da7-b28c-b715b2392948 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:04.584997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:04.585612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.585897 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.586174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.586820 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:04:04.587132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:04.588762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:04.600725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1201/2397] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:04:04.605276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe07dc-c804-406a-ae28-9b87dd7501d6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:04.614502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe07dc-c804-406a-ae28-9b87dd7501d6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:04.619122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:04.619706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.619981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.620287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.620905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:04.622776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:04.653176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1197/2398] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:04:04 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:04.654365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:04.665143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8d54e-52fc-4f4b-9a35-d76f62f72b3d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:04.683958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:04.728436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:04.729052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.729339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.729616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.730234 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:04.732211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:04.753896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1202/2399] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:04:04 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:04.758094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:04.768396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265933c-fbe7-428e-a784-87f7325a559f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:04.773446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:04.774038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.774316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.774584 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.775218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:04.776903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:04.787098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1198/2400] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:04:04.790880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:04.800971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c919d6-d7f9-4696-a900-7e776d3b0ba1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:04.805682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e358ea-617d-44bf-ba5f-adc971a0f40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:04.806311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e358ea-617d-44bf-ba5f-adc971a0f40e admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.806590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e358ea-617d-44bf-ba5f-adc971a0f40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.806862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e358ea-617d-44bf-ba5f-adc971a0f40e admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.807529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e358ea-617d-44bf-ba5f-adc971a0f40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:04.809209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e358ea-617d-44bf-ba5f-adc971a0f40e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:04.819217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1203/2401] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:04:04.823278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:04.833749 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcaaf4-83f8-4baa-a41a-02e1b728add5 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:04.838606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:04.839248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.839523 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.839795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] PATH_INFO: `/v3/projects/ab834d315d084239a710038047b85a72/users/45154585178a47b2b028ebdaa26fa602/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.857198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab834d315d084239a710038047b85a72, user_id=45154585178a47b2b028ebdaa26fa602, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:04.859836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:04.872840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1199/2402] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:04:04 2022] PUT /identity/v3/projects/ab834d315d084239a710038047b85a72/users/45154585178a47b2b028ebdaa26fa602/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:04.884853 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4306c1e-1654-4695-a643-db0d1637a398 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:04.885140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4306c1e-1654-4695-a643-db0d1637a398 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:04.885439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4306c1e-1654-4695-a643-db0d1637a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:04.914794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c4306c1e-1654-4695-a643-db0d1637a398 None None] MFA Rules not processed for user `45154585178a47b2b028ebdaa26fa602`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:04.916076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c4306c1e-1654-4695-a643-db0d1637a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:04.985400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1204/2403] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:04.994386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.054473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198bdd-fb61-4c52-b568-d170a7eb5e13 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.059071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.059674 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.059956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.060266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.060876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.062773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.076829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:05.095261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1200/2404] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:04:04 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:05.096401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4306c1e-1654-4695-a643-db0d1637a398 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.108521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4306c1e-1654-4695-a643-db0d1637a398 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.167125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.167655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.167940 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.168261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.168890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.170906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.193748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1205/2405] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:04:05 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:05.197740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.209329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acee41b-3dbd-4aa5-be7d-ac39f9bc4092 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.215507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.216162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.216453 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.216724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.217370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.218965 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.230166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1201/2406] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:04:05.234463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.244754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e1fcb-f9f7-4b90-b429-612b5b168b0b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.249737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.250403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.250684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.250958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] PATH_INFO: `/v3/projects/da3499bb796644728b3dd936115ad5e6/users/9b342c33da4a4774a8651b95db740635/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.266759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3499bb796644728b3dd936115ad5e6, user_id=9b342c33da4a4774a8651b95db740635, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.270032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.272349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:05.284498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1206/2407] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:04:05 2022] PUT /identity/v3/projects/da3499bb796644728b3dd936115ad5e6/users/9b342c33da4a4774a8651b95db740635/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:04:05.288594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.299032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4501708-3dc6-4a95-aa75-6e09467e1704 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.369013 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c825164-36da-4115-ab7b-e563f00961ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.369623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c825164-36da-4115-ab7b-e563f00961ce admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.369900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c825164-36da-4115-ab7b-e563f00961ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.370170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c825164-36da-4115-ab7b-e563f00961ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.370808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c825164-36da-4115-ab7b-e563f00961ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.372436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c825164-36da-4115-ab7b-e563f00961ce admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.384178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1202/2408] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:04:05.387960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.399755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca87923-b3e0-4376-8d5e-64c630c3427f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.405181 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.405871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.406147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.406421 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] PATH_INFO: `/v3/projects/da3499bb796644728b3dd936115ad5e6/users/9b342c33da4a4774a8651b95db740635/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.408378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3499bb796644728b3dd936115ad5e6, user_id=9b342c33da4a4774a8651b95db740635, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.411044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.421857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1207/2409] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:04:05 2022] PUT /identity/v3/projects/da3499bb796644728b3dd936115ad5e6/users/9b342c33da4a4774a8651b95db740635/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:05.433701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa584c-d7b0-4550-9e05-f853c8e2c46e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.433701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa584c-d7b0-4550-9e05-f853c8e2c46e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.433981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa584c-d7b0-4550-9e05-f853c8e2c46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.461940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-89aa584c-d7b0-4550-9e05-f853c8e2c46e None None] MFA Rules not processed for user `9b342c33da4a4774a8651b95db740635`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:05.463148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-89aa584c-d7b0-4550-9e05-f853c8e2c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:05.533818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1203/2410] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:05.547543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.549373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:05.608640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9bc4c-e42e-404d-81bd-23302f058527 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.612564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.613378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.613652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.613923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.623035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.624569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.634182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1208/2411] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:05.712124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa584c-d7b0-4550-9e05-f853c8e2c46e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:05.722430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa584c-d7b0-4550-9e05-f853c8e2c46e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:05.726558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301f1d7-21ec-4b78-9ffd-f7fea40f0cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:05.727215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301f1d7-21ec-4b78-9ffd-f7fea40f0cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:05.727499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301f1d7-21ec-4b78-9ffd-f7fea40f0cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:05.727791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301f1d7-21ec-4b78-9ffd-f7fea40f0cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:05.786650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301f1d7-21ec-4b78-9ffd-f7fea40f0cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:05.788892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301f1d7-21ec-4b78-9ffd-f7fea40f0cd3 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:05.799266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1204/2412] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:06.084269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:06.147228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20b26c-02d6-4cef-95fa-30aea301c828 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:06.151847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:06.152533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:06.152815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:06.153086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] PATH_INFO: `/v3/users/45154585178a47b2b028ebdaa26fa602` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:06.154263 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=45154585178a47b2b028ebdaa26fa602)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:06.156245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:06.216798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45154585178a47b2b028ebdaa26fa602'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:06.217659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45154585178a47b2b028ebdaa26fa602'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:06.227044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45154585178a47b2b028ebdaa26fa602'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:06.227723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45154585178a47b2b028ebdaa26fa602'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:06.240488 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45154585178a47b2b028ebdaa26fa602'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:06.253941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1209/2413] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:04:06 2022] DELETE /identity/v3/users/45154585178a47b2b028ebdaa26fa602 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:06.258372 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:06.258668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:06.258936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:06.265933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:06.279698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:06.280942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:06.353374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1205/2414] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:06.363026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:06.364592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:06.370261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:06.434117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b640d-55ad-4e2d-b7bc-34a65a7c0b1b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:06.438964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:06.439586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:06.439870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:06.440176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:06.450345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:06.451730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:06.463756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1210/2415] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:06.908820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:06.910340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:06.923441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe20cc-7f53-49f3-86e3-70624b5083fa None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:06.936073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:06.937220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:06.937507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:06.937781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] PATH_INFO: `/v3/projects/ab834d315d084239a710038047b85a72` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:06.939129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab834d315d084239a710038047b85a72)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:06.941791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:06.998154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab834d315d084239a710038047b85a72', 'request_id': 'req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.007804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab834d315d084239a710038047b85a72', 'request_id': 'req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.014403 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab834d315d084239a710038047b85a72', 'request_id': 'req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.028301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] The token cache is being invalidate because project ab834d315d084239a710038047b85a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:04:07.028301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.028935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.030540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1206/2416] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:04:06 2022] DELETE /identity/v3/projects/ab834d315d084239a710038047b85a72 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:04:07.034533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:07.044308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c4cc5-8211-497f-889c-a5356563fea9 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:07.119024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:07.119737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.119936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.120241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] PATH_INFO: `/v3/users/9b342c33da4a4774a8651b95db740635` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.121377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b342c33da4a4774a8651b95db740635)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:07.123284 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:07.171979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b342c33da4a4774a8651b95db740635'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.172657 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b342c33da4a4774a8651b95db740635'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.179873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b342c33da4a4774a8651b95db740635'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.180575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b342c33da4a4774a8651b95db740635'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.193128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0ead1c4a-0445-4377-bd01-6ea0bdf372fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b342c33da4a4774a8651b95db740635'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.205620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1211/2417] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:04:07 2022] DELETE /identity/v3/users/9b342c33da4a4774a8651b95db740635 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:07.496490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:07.498015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:07.517793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be72c8-07d6-455d-a8d2-7f1bb2bdc16f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:07.576228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:07.576786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.577590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.577870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] PATH_INFO: `/v3/projects/da3499bb796644728b3dd936115ad5e6` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.578976 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3499bb796644728b3dd936115ad5e6)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:07.580934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:07.611173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:07.627482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3499bb796644728b3dd936115ad5e6', 'request_id': 'req-46f3a824-f50c-413d-a4ea-f229642ab56c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.633516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3499bb796644728b3dd936115ad5e6', 'request_id': 'req-46f3a824-f50c-413d-a4ea-f229642ab56c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.639591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3499bb796644728b3dd936115ad5e6', 'request_id': 'req-46f3a824-f50c-413d-a4ea-f229642ab56c'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.652282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] The token cache is being invalidate because project da3499bb796644728b3dd936115ad5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:04:07.652573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.653194 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-46f3a824-f50c-413d-a4ea-f229642ab56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:07.654688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1207/2418] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:04:07 2022] DELETE /identity/v3/projects/da3499bb796644728b3dd936115ad5e6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:07.659683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7b97-8934-4bc7-9e7e-bb82ebc9cc80 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.659970 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7b97-8934-4bc7-9e7e-bb82ebc9cc80 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.660308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7b97-8934-4bc7-9e7e-bb82ebc9cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.680566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-fbdd7b97-8934-4bc7-9e7e-bb82ebc9cc80 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:07.681750 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-fbdd7b97-8934-4bc7-9e7e-bb82ebc9cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:07.700310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:07.740801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1212/2419] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:07.748803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cae408-a682-462e-9fdb-8647eb59677f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.749095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cae408-a682-462e-9fdb-8647eb59677f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.749373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cae408-a682-462e-9fdb-8647eb59677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.770299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-34cae408-a682-462e-9fdb-8647eb59677f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:07.771494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-34cae408-a682-462e-9fdb-8647eb59677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:07.779255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1208/2420] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:07.783424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd7b97-8934-4bc7-9e7e-bb82ebc9cc80 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:07.792911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd7b97-8934-4bc7-9e7e-bb82ebc9cc80 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:07.797707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:07.798315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.798595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.798866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.799510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:04:07.799812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:07.801996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:07.813291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1213/2421] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:04:07.818098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cae408-a682-462e-9fdb-8647eb59677f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:07.827553 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cae408-a682-462e-9fdb-8647eb59677f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:07.832192 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:07.832789 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.833095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.833369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.834026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:07.836213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:07.863214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1209/2422] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:04:07 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:07.867378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:07.877337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684829e-6d06-4d1d-a222-615497d1f69a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:07.935220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:07.935832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.936115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.936429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.937052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:07.938958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:07.963152 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1214/2423] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:04:07 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:07.967271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:07.978134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082787f-8755-4a8e-bb4a-c65342deb69b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:07.982773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:07.983366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:07.983640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:07.983907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:07.984571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:07.986147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:07.996419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1210/2424] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:04:08.000294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:08.011058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6463a-bc23-4a8f-8b87-48984db9ac3b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:08.015689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a1c1f-bf47-4e27-85d1-9f45508d8085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:08.016369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1c1f-bf47-4e27-85d1-9f45508d8085 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:08.016650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1c1f-bf47-4e27-85d1-9f45508d8085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:08.016920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1c1f-bf47-4e27-85d1-9f45508d8085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:08.017549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1c1f-bf47-4e27-85d1-9f45508d8085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:08.019739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1c1f-bf47-4e27-85d1-9f45508d8085 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:08.030343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1215/2425] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:04:08.034396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:08.044387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf54e9e-3ef9-4236-bb28-24b67bc52d62 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:08.049085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:08.049727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:08.050008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:08.050283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] PATH_INFO: `/v3/projects/5b7c7dd5bd484c119ac7796d91fde596/users/9cffeda34f4a4f7ab85189d26f25e9e6/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:08.064596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7c7dd5bd484c119ac7796d91fde596, user_id=9cffeda34f4a4f7ab85189d26f25e9e6, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:08.067876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:08.080714 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1211/2426] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:04:08 2022] PUT /identity/v3/projects/5b7c7dd5bd484c119ac7796d91fde596/users/9cffeda34f4a4f7ab85189d26f25e9e6/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:08.092753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea1dc5-9c40-491d-b801-a33298290234 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:08.093039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea1dc5-9c40-491d-b801-a33298290234 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:08.093312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea1dc5-9c40-491d-b801-a33298290234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:08.120317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9eea1dc5-9c40-491d-b801-a33298290234 None None] MFA Rules not processed for user `9cffeda34f4a4f7ab85189d26f25e9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:08.121509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9eea1dc5-9c40-491d-b801-a33298290234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:08.185362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1216/2427] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:08.196103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:08.197782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:08.254685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67360f1-ce8a-4c79-880c-f72b038b28dd admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:08.258321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b2634-dd7b-4760-a026-cc6b3d3856c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:08.258929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b2634-dd7b-4760-a026-cc6b3d3856c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:08.259203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b2634-dd7b-4760-a026-cc6b3d3856c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:08.259474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b2634-dd7b-4760-a026-cc6b3d3856c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:08.267657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b2634-dd7b-4760-a026-cc6b3d3856c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:08.269111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b2634-dd7b-4760-a026-cc6b3d3856c5 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:08.277786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1212/2428] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:12.325922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea1dc5-9c40-491d-b801-a33298290234 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:12.347119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:12.389917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea1dc5-9c40-491d-b801-a33298290234 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:12.394572 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:12.395212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:12.395494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:12.395784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] PATH_INFO: `/v3/users/9cffeda34f4a4f7ab85189d26f25e9e6` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:12.397626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cffeda34f4a4f7ab85189d26f25e9e6)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:12.399568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:12.446541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cffeda34f4a4f7ab85189d26f25e9e6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:12.447208 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cffeda34f4a4f7ab85189d26f25e9e6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:12.454550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cffeda34f4a4f7ab85189d26f25e9e6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:12.455170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cffeda34f4a4f7ab85189d26f25e9e6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:12.467799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cffeda34f4a4f7ab85189d26f25e9e6'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:12.479679 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1217/2429] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:04:12 2022] DELETE /identity/v3/users/9cffeda34f4a4f7ab85189d26f25e9e6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:12.484343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:12.484812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:12.484921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:12.503585 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:12.504808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:12.556790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:12.569196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1213/2430] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:12.579419 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:12.581557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:12.637964 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a09e-2e2c-4c4c-80fa-7a5406120e66 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:12.641622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33942dcc-cb3a-4fff-bc21-e3bc5c19270a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:12.642227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942dcc-cb3a-4fff-bc21-e3bc5c19270a service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:12.642504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942dcc-cb3a-4fff-bc21-e3bc5c19270a service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:12.642778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942dcc-cb3a-4fff-bc21-e3bc5c19270a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:12.652095 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33942dcc-cb3a-4fff-bc21-e3bc5c19270a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:12.653553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33942dcc-cb3a-4fff-bc21-e3bc5c19270a service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:12.663425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1218/2431] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:12.924962 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:12.926418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:12.940662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52bf40-4b0d-4c22-803c-1a59d2aa5e4d None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:12.946229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:12.946883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:12.947166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:12.947443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] PATH_INFO: `/v3/projects/5b7c7dd5bd484c119ac7796d91fde596` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:12.948666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7c7dd5bd484c119ac7796d91fde596)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:12.950532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:12.997734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7c7dd5bd484c119ac7796d91fde596', 'request_id': 'req-73fe4177-47de-4c98-bb9e-584bfbe72974'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:13.003542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7c7dd5bd484c119ac7796d91fde596', 'request_id': 'req-73fe4177-47de-4c98-bb9e-584bfbe72974'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:13.009739 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7c7dd5bd484c119ac7796d91fde596', 'request_id': 'req-73fe4177-47de-4c98-bb9e-584bfbe72974'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:13.022378 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] The token cache is being invalidate because project 5b7c7dd5bd484c119ac7796d91fde596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:04:13.022668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:13.023282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-73fe4177-47de-4c98-bb9e-584bfbe72974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:13.024782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1214/2432] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:04:12 2022] DELETE /identity/v3/projects/5b7c7dd5bd484c119ac7796d91fde596 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:13.029586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b0b7e-093c-41c1-93a3-daefca05d25b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.029878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b0b7e-093c-41c1-93a3-daefca05d25b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.030159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b0b7e-093c-41c1-93a3-daefca05d25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.049625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-532b0b7e-093c-41c1-93a3-daefca05d25b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:13.050817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-532b0b7e-093c-41c1-93a3-daefca05d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:13.087813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:13.110630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1219/2433] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:13.172957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27420fd-ea6b-4893-b46a-e498c7f8e65a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.173242 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27420fd-ea6b-4893-b46a-e498c7f8e65a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.173545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27420fd-ea6b-4893-b46a-e498c7f8e65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.194073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b27420fd-ea6b-4893-b46a-e498c7f8e65a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:13.195306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b27420fd-ea6b-4893-b46a-e498c7f8e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:13.203255 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1215/2434] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:13.207499 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b0b7e-093c-41c1-93a3-daefca05d25b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:13.218667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b0b7e-093c-41c1-93a3-daefca05d25b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:13.223202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:13.223802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.224106 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.224392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.225032 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:04:13.225335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:13.226933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:13.238244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1220/2435] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:04:13.242348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27420fd-ea6b-4893-b46a-e498c7f8e65a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:13.251652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27420fd-ea6b-4893-b46a-e498c7f8e65a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:13.256871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:13.257462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.257745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.258016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.258633 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:13.260569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:13.287110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1216/2436] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:04:13 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:13.291232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:13.301306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c0dd2-f606-4605-8539-92cc5e5bfc0f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:13.358210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:13.358880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.359077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.359346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.359994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:13.362081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:13.382712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1221/2437] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:04:13 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:04:13.386946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:13.397932 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3d54-7f69-4763-91bd-794e5295e78a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:13.402685 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:13.403282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.403562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.403836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.404511 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:13.406117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:13.416986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1217/2438] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:04:13.421055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:13.431559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb368c4-9986-4cc3-9b4b-e2201968b1cb admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:13.436426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8dfb6-da91-4436-9fdc-6a1e63b47cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:13.437027 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8dfb6-da91-4436-9fdc-6a1e63b47cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.437309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8dfb6-da91-4436-9fdc-6a1e63b47cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.437582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8dfb6-da91-4436-9fdc-6a1e63b47cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.438249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8dfb6-da91-4436-9fdc-6a1e63b47cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:13.439829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8dfb6-da91-4436-9fdc-6a1e63b47cd5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:13.449685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1222/2439] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:04:13.453820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:13.464172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96899e3e-276b-4861-9e0b-2140822c04bf admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:13.468870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:13.469514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.469799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.470077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] PATH_INFO: `/v3/projects/f6ac5f65d14a4ff0ac68e18b017dc806/users/ef48209951a74c9183e438615081e0ef/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.485285 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ac5f65d14a4ff0ac68e18b017dc806, user_id=ef48209951a74c9183e438615081e0ef, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:13.487968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:13.501208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1218/2440] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:04:13 2022] PUT /identity/v3/projects/f6ac5f65d14a4ff0ac68e18b017dc806/users/ef48209951a74c9183e438615081e0ef/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:13.511690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595f042-ea12-4b38-81bc-ebaaadb4e3a0 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.512118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595f042-ea12-4b38-81bc-ebaaadb4e3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.512534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595f042-ea12-4b38-81bc-ebaaadb4e3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.540602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6595f042-ea12-4b38-81bc-ebaaadb4e3a0 None None] MFA Rules not processed for user `ef48209951a74c9183e438615081e0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:13.541808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6595f042-ea12-4b38-81bc-ebaaadb4e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:13.604906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1223/2441] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:13.617187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:13.618879 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:13.676304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b1dcd-f7ee-495b-aa2c-a95912c6dd9d admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:13.680469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c76a4a9747469d88a0a1a5748452a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:13.681082 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:13.681363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:13.681638 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:13.689911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:13.691345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:13.700792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1219/2442] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:23.602120 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:23.697176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:41.750343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:41.750979 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:41.751257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:41.792978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] MFA Rules not processed for user `7a701149974b45db9f782bc5a232aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:41.794191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:41.848351 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1224/2443] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:44.660300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:44.661802 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:44.718312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790389eb-73d7-429b-b649-858e858952b3 service glance] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:44.721993 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:44.722593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:44.722869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:44.723159 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:44.724842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:44.794476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:44.795881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:44.806577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1220/2444] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:45.914803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:45.916846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:45.930155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23feb6-345a-4866-bc93-91c591ec17ac None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:45.933776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892d868-b4d0-408f-bd95-cf5dbe74d521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:45.934386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892d868-b4d0-408f-bd95-cf5dbe74d521 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:45.934667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892d868-b4d0-408f-bd95-cf5dbe74d521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:45.934941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892d868-b4d0-408f-bd95-cf5dbe74d521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:45.936776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c892d868-b4d0-408f-bd95-cf5dbe74d521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:45.950174 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892d868-b4d0-408f-bd95-cf5dbe74d521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:45.951607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892d868-b4d0-408f-bd95-cf5dbe74d521 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:45.962709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1225/2445] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:04:50.935087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:51.001104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac40b-8356-495a-9e0f-914ffe2d7175 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:51.005690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:51.006315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:51.006590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:51.006856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] PATH_INFO: `/v3/users/ef48209951a74c9183e438615081e0ef` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:51.007975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef48209951a74c9183e438615081e0ef)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:51.009895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:51.052483 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:51.055430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef48209951a74c9183e438615081e0ef'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.063921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef48209951a74c9183e438615081e0ef'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.070925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef48209951a74c9183e438615081e0ef'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.071676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef48209951a74c9183e438615081e0ef'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.084044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef48209951a74c9183e438615081e0ef'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.091149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1221/2446] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:04:50 2022] DELETE /identity/v3/users/ef48209951a74c9183e438615081e0ef => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:51.100654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:51.101179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:51.101179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:51.119528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:04:51.121265 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:51.179842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1226/2447] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:04:51.190107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:51.191571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:51.255618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914118ed-2f37-4ccb-bb36-0eaa2e9c5f04 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:51.259235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:51.259835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:51.260109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:51.260402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:51.269390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:51.270873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:51.280418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1222/2448] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:04:51.560344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:51.561805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:51.573984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ba209-ab74-4863-a416-239b1f1dcc70 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:51.578843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:51.579486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:51.579766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:51.580066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] PATH_INFO: `/v3/projects/f6ac5f65d14a4ff0ac68e18b017dc806` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:51.581224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ac5f65d14a4ff0ac68e18b017dc806)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:51.583184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:51.630376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ac5f65d14a4ff0ac68e18b017dc806', 'request_id': 'req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.635783 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ac5f65d14a4ff0ac68e18b017dc806', 'request_id': 'req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.641794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ac5f65d14a4ff0ac68e18b017dc806', 'request_id': 'req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.645377 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:51.656759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] The token cache is being invalidate because project f6ac5f65d14a4ff0ac68e18b017dc806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:04:51.657280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.658240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:04:51.659765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1227/2449] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:04:51 2022] DELETE /identity/v3/projects/f6ac5f65d14a4ff0ac68e18b017dc806 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:04:56.660518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:04:56.661828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:56.734886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f614638-230e-4486-a4b4-ff033fc81f61 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:04:56.787588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:04:56.800315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b60ba-8d25-484a-acd5-31ff56b2d8cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:04:56.800930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b60ba-8d25-484a-acd5-31ff56b2d8cd service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:04:56.801204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b60ba-8d25-484a-acd5-31ff56b2d8cd service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:04:56.801473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b60ba-8d25-484a-acd5-31ff56b2d8cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:04:56.803195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-845b60ba-8d25-484a-acd5-31ff56b2d8cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:04:56.861868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b60ba-8d25-484a-acd5-31ff56b2d8cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:04:56.863455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b60ba-8d25-484a-acd5-31ff56b2d8cd service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:04:56.882262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1223/2450] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:04:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:01.658451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:01.743695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:01.745147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:01.760941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bbb9-c6b1-4389-a29f-2262ffeb2326 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:01.766910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2fd80-4a92-41f4-adaa-c8fd0890e72a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:01.767509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2fd80-4a92-41f4-adaa-c8fd0890e72a service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:01.767796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2fd80-4a92-41f4-adaa-c8fd0890e72a service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:01.768068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2fd80-4a92-41f4-adaa-c8fd0890e72a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:01.769850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b4c2fd80-4a92-41f4-adaa-c8fd0890e72a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:01.849505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2fd80-4a92-41f4-adaa-c8fd0890e72a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:01.851101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2fd80-4a92-41f4-adaa-c8fd0890e72a service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:01.868995 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1228/2451] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:06.879477 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=32 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:07.678500 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4c30-1c0e-40b7-8781-541f0ada6bc8 None None] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:07.678808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4c30-1c0e-40b7-8781-541f0ada6bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:07.679081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4c30-1c0e-40b7-8781-541f0ada6bc8 None None] PATH_INFO: `/` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:07.680507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1224/2452] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 17:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:05:07.687596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6109d0-4d93-4553-9884-2a555a56ef29 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:07.688122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6109d0-4d93-4553-9884-2a555a56ef29 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:07.688272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6109d0-4d93-4553-9884-2a555a56ef29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:07.739651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-cb6109d0-4d93-4553-9884-2a555a56ef29 None None] MFA Rules not processed for user `256af8436dd842fdb4ebe90ec9be26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:07.740917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-cb6109d0-4d93-4553-9884-2a555a56ef29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:07.803462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1229/2453] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 17:05:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:05:07.825206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4c30-1c0e-40b7-8781-541f0ada6bc8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:07.828421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e02c4c30-1c0e-40b7-8781-541f0ada6bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:07.842450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4c30-1c0e-40b7-8781-541f0ada6bc8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:07.846697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5bdab-7955-46a5-9ff4-408f5107cd0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:07.847311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5bdab-7955-46a5-9ff4-408f5107cd0d service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:07.847589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5bdab-7955-46a5-9ff4-408f5107cd0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:07.847859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5bdab-7955-46a5-9ff4-408f5107cd0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:07.855787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5bdab-7955-46a5-9ff4-408f5107cd0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:07.857755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5bdab-7955-46a5-9ff4-408f5107cd0d service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:07.867950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1225/2454] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:07.982700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9952-4449-42a9-9a53-bbf14352aead None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:07.982986 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9952-4449-42a9-9a53-bbf14352aead None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:07.983305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9952-4449-42a9-9a53-bbf14352aead None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:07.984739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1230/2455] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 17:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:05:07.990619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66d55-a03c-476f-b812-babfb1b21da5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:07.991101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66d55-a03c-476f-b812-babfb1b21da5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:07.991309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66d55-a03c-476f-b812-babfb1b21da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:08.033612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-53c66d55-a03c-476f-b812-babfb1b21da5 None None] MFA Rules not processed for user `86456c6dbe4e454e9f255960b70c0832`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:08.034745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-53c66d55-a03c-476f-b812-babfb1b21da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:08.045569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1226/2456] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 17:05:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:05:08.056230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9952-4449-42a9-9a53-bbf14352aead None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:08.068641 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9952-4449-42a9-9a53-bbf14352aead None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:08.072244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:08.072849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:08.073128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:08.073403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:08.084779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:08.086541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:08.097838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1231/2457] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:13.052582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f126b1d-1b8e-4f72-a142-f6c7a9cbe8ae None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:13.053062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f126b1d-1b8e-4f72-a142-f6c7a9cbe8ae None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:13.053163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f126b1d-1b8e-4f72-a142-f6c7a9cbe8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:13.105706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-8f126b1d-1b8e-4f72-a142-f6c7a9cbe8ae None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:13.106946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8f126b1d-1b8e-4f72-a142-f6c7a9cbe8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:13.169352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1227/2458] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 17:05:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:05:13.214886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:13.216317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:13.263112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:13.306547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841693d-e2f3-4b9a-9f31-8258963b669f service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:13.311654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f18b2-7c66-4390-8353-d3997f7e8966 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:13.312333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f18b2-7c66-4390-8353-d3997f7e8966 service swift] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:13.312612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f18b2-7c66-4390-8353-d3997f7e8966 service swift] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:13.312883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f18b2-7c66-4390-8353-d3997f7e8966 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:13.320682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f18b2-7c66-4390-8353-d3997f7e8966 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:13.322253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f18b2-7c66-4390-8353-d3997f7e8966 service swift] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:13.330094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1232/2459] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:05:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:05:23.166527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=55 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:23.327654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=21 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:25.412899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f126b1d-1b8e-4f72-a142-f6c7a9cbe8ae None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:25.422296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f126b1d-1b8e-4f72-a142-f6c7a9cbe8ae None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:25.425915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:25.426516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:25.426794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:25.427064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:25.428813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:25.439448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:25.440903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:25.449580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1228/2460] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:30.929774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8950e-fe19-4a7e-b1dc-92a9bc582e14 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:30.930243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8950e-fe19-4a7e-b1dc-92a9bc582e14 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:30.930460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8950e-fe19-4a7e-b1dc-92a9bc582e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:30.949722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d4d8950e-fe19-4a7e-b1dc-92a9bc582e14 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:30.950845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d4d8950e-fe19-4a7e-b1dc-92a9bc582e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:30.957893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1233/2461] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 17:05:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:05:30.969918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:30.981839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5368c-7188-4adb-a9e4-154ad484ac64 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:30.987583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:30.988211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:30.988493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:30.988763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:30.996784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:30.998367 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:31.006850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1229/2462] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:05:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:05:40.944622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:41.003778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=12 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:45.173701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8950e-fe19-4a7e-b1dc-92a9bc582e14 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:45.187238 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8950e-fe19-4a7e-b1dc-92a9bc582e14 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:45.192811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee264dfc-83af-4857-b4ca-37d41ce65ad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:45.193462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264dfc-83af-4857-b4ca-37d41ce65ad7 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:45.193747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264dfc-83af-4857-b4ca-37d41ce65ad7 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:45.194041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264dfc-83af-4857-b4ca-37d41ce65ad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:45.195990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ee264dfc-83af-4857-b4ca-37d41ce65ad7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:45.280704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee264dfc-83af-4857-b4ca-37d41ce65ad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:45.282821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee264dfc-83af-4857-b4ca-37d41ce65ad7 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:45.301029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1234/2463] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:49.610383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:49.612042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:49.629613 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d5f64-e4c1-48f5-9d14-96bb33c709c0 service swift] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:49.635293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:49.635898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:49.636202 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:49.636479 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:49.638223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:49.650502 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:49.652078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:49.662974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1230/2464] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:51.651636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:51.652209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:51.652209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:51.670996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:51.672254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:51.679610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1235/2465] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 10 17:05:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:05:51.692622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:51.704618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d205e-1ab7-4d47-8765-1cf94e24b916 service nova] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:51.709528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:51.710380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:51.710658 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:51.710924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:51.718760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:51.720400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:51.728381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1231/2466] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:05:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:06:01.666142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=44 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:01.725154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=12 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:14.186745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:14.188220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:14.244505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837416-b918-4cf9-b08d-bf8c3a77a29b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:14.248325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bee95-179b-4ce4-b254-2f4df72162f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:14.248944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bee95-179b-4ce4-b254-2f4df72162f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:14.249258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bee95-179b-4ce4-b254-2f4df72162f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:14.249534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bee95-179b-4ce4-b254-2f4df72162f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:14.251254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b29bee95-179b-4ce4-b254-2f4df72162f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:14.312262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bee95-179b-4ce4-b254-2f4df72162f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:14.313714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bee95-179b-4ce4-b254-2f4df72162f3 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:14.323806 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1236/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:20.557110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:20.558510 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:20.571045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d20b8e-a7b3-48c2-ad9b-14f4b8720864 service swift] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:20.574706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591a84f-537f-4b5a-b834-900b60b36f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:20.575312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591a84f-537f-4b5a-b834-900b60b36f86 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:20.575596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591a84f-537f-4b5a-b834-900b60b36f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:20.575873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591a84f-537f-4b5a-b834-900b60b36f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:20.577630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8591a84f-537f-4b5a-b834-900b60b36f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:20.640778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591a84f-537f-4b5a-b834-900b60b36f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:20.642227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591a84f-537f-4b5a-b834-900b60b36f86 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:20.652682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1232/2468] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:23.923759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4694e5-b30f-466c-ba4c-031799a97ee6 None None] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:23.924255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4694e5-b30f-466c-ba4c-031799a97ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:23.924376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4694e5-b30f-466c-ba4c-031799a97ee6 None None] PATH_INFO: `/` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:23.925708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1237/2469] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 10 17:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:23.959026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:23.959573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:23.959910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:23.997704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4694e5-b30f-466c-ba4c-031799a97ee6 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:23.999566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3d4694e5-b30f-466c-ba4c-031799a97ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.007201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] MFA Rules not processed for user `836987c3b0a9483db61d8f31ad2af00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:24.008443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.017354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4694e5-b30f-466c-ba4c-031799a97ee6 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.022619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.023250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.023531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.023807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] PATH_INFO: `/v3/users/77b8847d10744ec98e0ed7a3f1e991ed` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.025012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b8847d10744ec98e0ed7a3f1e991ed)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.026946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.076360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b8847d10744ec98e0ed7a3f1e991ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.077322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b8847d10744ec98e0ed7a3f1e991ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.077468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1233/2470] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 10 17:06:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:06:24.086143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b8847d10744ec98e0ed7a3f1e991ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.086837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b8847d10744ec98e0ed7a3f1e991ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.089622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.091185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.102233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b8847d10744ec98e0ed7a3f1e991ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.104822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045337ba-67d2-4d15-a823-539d79be93dd None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.109945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d29f50-ad7b-4145-a150-97b03f1e63e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.110742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d29f50-ad7b-4145-a150-97b03f1e63e7 service placement] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.111008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d29f50-ad7b-4145-a150-97b03f1e63e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.111277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d29f50-ad7b-4145-a150-97b03f1e63e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.113060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-13d29f50-ad7b-4145-a150-97b03f1e63e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.114729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1238/2471] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:06:23 2022] DELETE /identity/v3/users/77b8847d10744ec98e0ed7a3f1e991ed => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:06:24.125561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d29f50-ad7b-4145-a150-97b03f1e63e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.127144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d29f50-ad7b-4145-a150-97b03f1e63e7 service placement] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.136933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1234/2472] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:06:24.294770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.296306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.331936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:24.368218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a38331-35f2-45a6-89e3-00d28f9487be admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.372845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.373416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.373693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.373962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] PATH_INFO: `/v3/projects/c11dfd80b6c544f6b87a58d5a25c5649` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.375068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c11dfd80b6c544f6b87a58d5a25c5649)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.377292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.423084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c11dfd80b6c544f6b87a58d5a25c5649', 'request_id': 'req-03442940-622e-48eb-9c6f-b16da00ad02a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.428845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c11dfd80b6c544f6b87a58d5a25c5649', 'request_id': 'req-03442940-622e-48eb-9c6f-b16da00ad02a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.435201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c11dfd80b6c544f6b87a58d5a25c5649', 'request_id': 'req-03442940-622e-48eb-9c6f-b16da00ad02a'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.448318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] The token cache is being invalidate because project c11dfd80b6c544f6b87a58d5a25c5649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:06:24.448608 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.449215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-03442940-622e-48eb-9c6f-b16da00ad02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:24.450760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1239/2473] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:06:24 2022] DELETE /identity/v3/projects/c11dfd80b6c544f6b87a58d5a25c5649 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:24.455783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e92ad-a621-408c-8723-f3e850c08de5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.456070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e92ad-a621-408c-8723-f3e850c08de5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.456365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e92ad-a621-408c-8723-f3e850c08de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.475981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d34e92ad-a621-408c-8723-f3e850c08de5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:24.477213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d34e92ad-a621-408c-8723-f3e850c08de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.535493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1235/2474] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:24.544540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ace236-5c4d-4922-ae7f-fece2504c04c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.544828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ace236-5c4d-4922-ae7f-fece2504c04c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.545101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ace236-5c4d-4922-ae7f-fece2504c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.565232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-89ace236-5c4d-4922-ae7f-fece2504c04c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:24.566418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-89ace236-5c4d-4922-ae7f-fece2504c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.575223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1240/2475] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:06:24.579380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e92ad-a621-408c-8723-f3e850c08de5 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.586659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:24.590517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e92ad-a621-408c-8723-f3e850c08de5 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.595390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.595992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.596316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.596592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.597241 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:06:24.597610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.599215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.610768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1236/2476] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:06:24.615058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ace236-5c4d-4922-ae7f-fece2504c04c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.624218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ace236-5c4d-4922-ae7f-fece2504c04c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.628817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.629909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.630183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.630448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.631054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.632961 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.668954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1241/2477] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:06:24 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:24.672938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.683532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8951f-4fbf-4d84-9949-89104e4133cc admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.737978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.738577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.739355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.739628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.740297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.742197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.763586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1237/2478] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:06:24 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:24.767858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.777990 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e064e-56cc-4cbe-b0cf-b9066ef5542f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.782544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.783160 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.783416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.783674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.784335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.785931 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.795458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1242/2479] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:24.799284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.809903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939e228-d0fc-4f51-b562-42c349dac651 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.814624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabc813-217a-4e10-b3d0-73b5f96f7774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.815215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabc813-217a-4e10-b3d0-73b5f96f7774 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.815497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabc813-217a-4e10-b3d0-73b5f96f7774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.815774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabc813-217a-4e10-b3d0-73b5f96f7774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.816448 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabc813-217a-4e10-b3d0-73b5f96f7774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.818665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabc813-217a-4e10-b3d0-73b5f96f7774 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.829143 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1238/2480] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:24.833146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:24.843117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423102f1-2793-460d-bd4c-e3c06e7b1f8c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:24.847626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:24.848369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.848650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.848919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] PATH_INFO: `/v3/projects/ceda1ad09e414f15888367d380d3cfb4/users/92c4f10067c64186816917ff982dcfe1/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.863028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceda1ad09e414f15888367d380d3cfb4, user_id=92c4f10067c64186816917ff982dcfe1, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:24.866172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:24.878652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1243/2481] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:24 2022] PUT /identity/v3/projects/ceda1ad09e414f15888367d380d3cfb4/users/92c4f10067c64186816917ff982dcfe1/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:06:24.891081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b702971-95f9-473e-8c4b-60d8a938d8fa None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:24.891370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b702971-95f9-473e-8c4b-60d8a938d8fa None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:24.891654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b702971-95f9-473e-8c4b-60d8a938d8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:24.919779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-9b702971-95f9-473e-8c4b-60d8a938d8fa None None] MFA Rules not processed for user `92c4f10067c64186816917ff982dcfe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:24.921019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9b702971-95f9-473e-8c4b-60d8a938d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:24.986101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1239/2482] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:24.991078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:25.036401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:25.055062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1503ad-bcaf-4b68-a9c8-6614eaa3614f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:25.060257 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:25.060904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.061166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.061441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.062076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:25.063988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:25.091673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1244/2483] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:06:24 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:25.096089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b702971-95f9-473e-8c4b-60d8a938d8fa None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:25.107641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b702971-95f9-473e-8c4b-60d8a938d8fa None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:25.162956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:25.163492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.163773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.164047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.164706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:25.166635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:25.182657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:25.198877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1240/2484] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:06:25 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:25.203170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:25.213676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f3b74-a1a2-4004-9cb5-896081c8ceb7 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:25.218319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:25.218920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.219198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.219469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.220105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:25.221722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:25.232558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1245/2485] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:25.236892 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:25.247626 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254aa7c-d163-478c-b1dd-7ea8018214c1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:25.252295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:25.252946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.253222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.253491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] PATH_INFO: `/v3/projects/a58e62c4157847baa85841dd24175857/users/53fcdce6588f4372a14fee2e853aef6a/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.271679 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58e62c4157847baa85841dd24175857, user_id=53fcdce6588f4372a14fee2e853aef6a, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:25.274419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:25.289671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1241/2486] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:25 2022] PUT /identity/v3/projects/a58e62c4157847baa85841dd24175857/users/53fcdce6588f4372a14fee2e853aef6a/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:25.293557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:25.306562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e160-ec1b-4da5-a567-08eda1556ca3 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:25.366650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0314a-03fa-479f-8ee9-f0914518a637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:25.367296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0314a-03fa-479f-8ee9-f0914518a637 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.367574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0314a-03fa-479f-8ee9-f0914518a637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.367845 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0314a-03fa-479f-8ee9-f0914518a637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.368518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0314a-03fa-479f-8ee9-f0914518a637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:25.370110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0314a-03fa-479f-8ee9-f0914518a637 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:25.380330 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1246/2487] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:06:25.384694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:25.395940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36a06-f0c9-439f-9a95-157d779ad1f3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:25.400970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:25.401659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.401970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.402245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] PATH_INFO: `/v3/projects/a58e62c4157847baa85841dd24175857/users/53fcdce6588f4372a14fee2e853aef6a/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.404237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58e62c4157847baa85841dd24175857, user_id=53fcdce6588f4372a14fee2e853aef6a, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:25.406914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:25.418145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1242/2488] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:25 2022] PUT /identity/v3/projects/a58e62c4157847baa85841dd24175857/users/53fcdce6588f4372a14fee2e853aef6a/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:25.430501 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5fc52-f725-41e8-892b-b2a8f6efe5b1 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.430819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5fc52-f725-41e8-892b-b2a8f6efe5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.431096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5fc52-f725-41e8-892b-b2a8f6efe5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.458564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-1db5fc52-f725-41e8-892b-b2a8f6efe5b1 None None] MFA Rules not processed for user `53fcdce6588f4372a14fee2e853aef6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:25.459776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1db5fc52-f725-41e8-892b-b2a8f6efe5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:25.527751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1247/2489] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:25.540412 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:25.542191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:25.612953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77470a-a513-468f-965b-4b7a1bc1e521 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:25.616847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32f5b8-97a0-42f5-bdfa-2455c5744b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:25.617457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32f5b8-97a0-42f5-bdfa-2455c5744b3e service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:25.617737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32f5b8-97a0-42f5-bdfa-2455c5744b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:25.618010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32f5b8-97a0-42f5-bdfa-2455c5744b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:25.627029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32f5b8-97a0-42f5-bdfa-2455c5744b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:25.628504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32f5b8-97a0-42f5-bdfa-2455c5744b3e service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:25.637995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1243/2490] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:26.031426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5fc52-f725-41e8-892b-b2a8f6efe5b1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:26.072043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:26.093100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5fc52-f725-41e8-892b-b2a8f6efe5b1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:26.098610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:26.099236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:26.099535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:26.099830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] PATH_INFO: `/v3/users/92c4f10067c64186816917ff982dcfe1` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:26.101009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c4f10067c64186816917ff982dcfe1)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:26.102913 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:26.147630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c4f10067c64186816917ff982dcfe1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.148316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c4f10067c64186816917ff982dcfe1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.155272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c4f10067c64186816917ff982dcfe1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.155878 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c4f10067c64186816917ff982dcfe1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.167558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c4f10067c64186816917ff982dcfe1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.180099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1248/2491] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:06:26 2022] DELETE /identity/v3/users/92c4f10067c64186816917ff982dcfe1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:06:26.184614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d601-3ce4-41f1-8325-ece9c8ddf083 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:26.185075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d601-3ce4-41f1-8325-ece9c8ddf083 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:26.185190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d601-3ce4-41f1-8325-ece9c8ddf083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:26.204977 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a2c7d601-3ce4-41f1-8325-ece9c8ddf083 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:26.206162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a2c7d601-3ce4-41f1-8325-ece9c8ddf083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:26.267345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1244/2492] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:26.276948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:26.278326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:26.335741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea581d64-d856-4ec6-8bdf-72fc38f83aeb admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:26.339426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:26.340052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:26.340424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:26.340728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:26.350124 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:26.351563 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:26.361792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1249/2493] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:26.432332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e3c5a-163a-426b-8db3-88b9a2473626 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:26.432580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e3c5a-163a-426b-8db3-88b9a2473626 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:26.432812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e3c5a-163a-426b-8db3-88b9a2473626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:26.452089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c08e3c5a-163a-426b-8db3-88b9a2473626 None None] MFA Rules not processed for user `a1c78f05a0f24151a1884faed65d286e`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:26.453326 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c08e3c5a-163a-426b-8db3-88b9a2473626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:26.503275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:26.516340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1245/2494] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:26.699724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:26.701212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:26.714551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ffdb5-fb50-4d45-8999-109fc8409db1 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:26.719132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:26.719757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:26.720043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:26.720903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] PATH_INFO: `/v3/projects/ceda1ad09e414f15888367d380d3cfb4` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:26.721479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceda1ad09e414f15888367d380d3cfb4)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:26.723902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:26.780071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceda1ad09e414f15888367d380d3cfb4', 'request_id': 'req-26795858-a953-426e-89d8-96e7e189b05f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.785157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:26.787270 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceda1ad09e414f15888367d380d3cfb4', 'request_id': 'req-26795858-a953-426e-89d8-96e7e189b05f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.796598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceda1ad09e414f15888367d380d3cfb4', 'request_id': 'req-26795858-a953-426e-89d8-96e7e189b05f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.816003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] The token cache is being invalidate because project ceda1ad09e414f15888367d380d3cfb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:06:26.816333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.816998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.818486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1250/2495] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:06:26 2022] DELETE /identity/v3/projects/ceda1ad09e414f15888367d380d3cfb4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:26.822521 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e3c5a-163a-426b-8db3-88b9a2473626 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:26.833540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e3c5a-163a-426b-8db3-88b9a2473626 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:26.891876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:26.892566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:26.892850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:26.893125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] PATH_INFO: `/v3/users/53fcdce6588f4372a14fee2e853aef6a` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:26.895332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53fcdce6588f4372a14fee2e853aef6a)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:26.896283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:26.944184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53fcdce6588f4372a14fee2e853aef6a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.951591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53fcdce6588f4372a14fee2e853aef6a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.961245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53fcdce6588f4372a14fee2e853aef6a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.962010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53fcdce6588f4372a14fee2e853aef6a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.975671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53fcdce6588f4372a14fee2e853aef6a'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:26.981868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1246/2496] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:06:26 2022] DELETE /identity/v3/users/53fcdce6588f4372a14fee2e853aef6a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:27.256930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.259522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:27.279070 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26795858-a953-426e-89d8-96e7e189b05f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.360065 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.361952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:27.368494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.369133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.369406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.369674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] PATH_INFO: `/v3/projects/a58e62c4157847baa85841dd24175857` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.370989 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58e62c4157847baa85841dd24175857)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.372830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.437262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58e62c4157847baa85841dd24175857', 'request_id': 'req-0d8216b3-6698-4494-aa31-f1c62f42ee18'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:27.439032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754ea73-8c96-4a1d-977c-1942512493a9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.443231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58e62c4157847baa85841dd24175857', 'request_id': 'req-0d8216b3-6698-4494-aa31-f1c62f42ee18'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:27.450277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58e62c4157847baa85841dd24175857', 'request_id': 'req-0d8216b3-6698-4494-aa31-f1c62f42ee18'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:27.458858 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:27.464101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] The token cache is being invalidate because project a58e62c4157847baa85841dd24175857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:06:27.464573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:27.465215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0d8216b3-6698-4494-aa31-f1c62f42ee18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:27.467010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1251/2497] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:06:27 2022] DELETE /identity/v3/projects/a58e62c4157847baa85841dd24175857 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:27.471818 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01e7f-351b-49ba-8562-bc1c17d81def None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.472400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01e7f-351b-49ba-8562-bc1c17d81def None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.472685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01e7f-351b-49ba-8562-bc1c17d81def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.491968 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-2ae01e7f-351b-49ba-8562-bc1c17d81def None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:27.493211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2ae01e7f-351b-49ba-8562-bc1c17d81def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:27.505702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacc4e8-ffbd-414f-bd39-522a3f7c6853 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.506314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacc4e8-ffbd-414f-bd39-522a3f7c6853 service placement] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.506594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacc4e8-ffbd-414f-bd39-522a3f7c6853 service placement] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.506865 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacc4e8-ffbd-414f-bd39-522a3f7c6853 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.508611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1dacc4e8-ffbd-414f-bd39-522a3f7c6853 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:27.519563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacc4e8-ffbd-414f-bd39-522a3f7c6853 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.521833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacc4e8-ffbd-414f-bd39-522a3f7c6853 service placement] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.532284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1247/2498] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:27.554268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1252/2499] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:06:27.563365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e50750-ed0b-4dac-ba44-40cb9ea8ad66 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.563657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e50750-ed0b-4dac-ba44-40cb9ea8ad66 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.563934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e50750-ed0b-4dac-ba44-40cb9ea8ad66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.584504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-d5e50750-ed0b-4dac-ba44-40cb9ea8ad66 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:27.585991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d5e50750-ed0b-4dac-ba44-40cb9ea8ad66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:27.594660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1248/2500] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:27.598984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae01e7f-351b-49ba-8562-bc1c17d81def None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.609050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae01e7f-351b-49ba-8562-bc1c17d81def None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.613937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.614550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.614832 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.615105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.615785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:06:27.616084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.617790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.630065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1253/2501] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:06:27.634348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e50750-ed0b-4dac-ba44-40cb9ea8ad66 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.646032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e50750-ed0b-4dac-ba44-40cb9ea8ad66 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.650795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.651392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.651666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.651938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.652677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.654573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.685974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1249/2502] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:06:27 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:27.690254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.696528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:27.703052 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ded6e-fb32-4cfc-b861-5de8d7c5a6a8 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.764080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.764743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.765022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.765293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.765916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.767884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.790574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1254/2503] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:06:27 2022] POST /identity/v3/users => generated 279 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:06:27.794732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.805730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50ae12-37d8-48bc-b00d-c29582800ce6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.810731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.811335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.811611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.811880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.812566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.814191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.825283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1250/2504] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:06:27.829210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.839547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb769f-977e-44f1-924f-58e52290f4f0 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.844992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183e370-2938-4b67-b6b5-4b5f7e3b998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.845601 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e370-2938-4b67-b6b5-4b5f7e3b998f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.845881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e370-2938-4b67-b6b5-4b5f7e3b998f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.846151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e370-2938-4b67-b6b5-4b5f7e3b998f admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.846819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183e370-2938-4b67-b6b5-4b5f7e3b998f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.848471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183e370-2938-4b67-b6b5-4b5f7e3b998f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.859547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1255/2505] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:27.865061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:27.880516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbde2d9-e98c-4b11-a645-93735f08ec1f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:27.885355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:27.886033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.886313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.886588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] PATH_INFO: `/v3/projects/c3ff15d6908d401ea09d6db16dbc2847/users/62a5a274301846f8ad502984e8fb826b/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.902588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff15d6908d401ea09d6db16dbc2847, user_id=62a5a274301846f8ad502984e8fb826b, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:27.905297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:27.920188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1251/2506] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:27 2022] PUT /identity/v3/projects/c3ff15d6908d401ea09d6db16dbc2847/users/62a5a274301846f8ad502984e8fb826b/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:27.932060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861594a-adc8-4cb0-9cf0-c8a1a024f22a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:27.932369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861594a-adc8-4cb0-9cf0-c8a1a024f22a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:27.932638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861594a-adc8-4cb0-9cf0-c8a1a024f22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:27.962315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-6861594a-adc8-4cb0-9cf0-c8a1a024f22a None None] MFA Rules not processed for user `62a5a274301846f8ad502984e8fb826b`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:27.963556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-6861594a-adc8-4cb0-9cf0-c8a1a024f22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:28.028677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1256/2507] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:28.033690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.099413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee2a0-4226-4e2f-bf4a-8769b085737a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.105212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.105940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.106211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.106480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.107142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.109420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.138393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1252/2508] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:06:28 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:28.142430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861594a-adc8-4cb0-9cf0-c8a1a024f22a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.153466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861594a-adc8-4cb0-9cf0-c8a1a024f22a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.210942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.211548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.211827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.212098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.212793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.214644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.231225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:28.236097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1257/2509] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:06:28 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:28.240911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.251262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f5eb4-77fa-471f-8627-43c94df35308 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.255927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.256562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.256845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.257116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.257794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.259411 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.271861 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1253/2510] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:28.276196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.286792 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952b4ac-640a-41ba-8a15-eb296514ad19 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.291398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.292043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.292349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.292621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] PATH_INFO: `/v3/projects/ae6011243d1a475caf5bb329af1cd4c2/users/a97fadd45aa549328c8bbbb538f23822/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.307380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6011243d1a475caf5bb329af1cd4c2, user_id=a97fadd45aa549328c8bbbb538f23822, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.310041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.322682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1258/2511] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:28 2022] PUT /identity/v3/projects/ae6011243d1a475caf5bb329af1cd4c2/users/a97fadd45aa549328c8bbbb538f23822/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:28.326792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.337109 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e669a0a-3ccf-451e-9fe3-e46eb02efbea admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.355391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:28.395529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2edee8-c8a0-40e3-abad-1361a16997e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.396166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2edee8-c8a0-40e3-abad-1361a16997e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.396425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2edee8-c8a0-40e3-abad-1361a16997e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.396699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2edee8-c8a0-40e3-abad-1361a16997e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.397330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2edee8-c8a0-40e3-abad-1361a16997e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.398902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2edee8-c8a0-40e3-abad-1361a16997e8 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.412368 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1254/2512] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:28.413618 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.424130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038c0fc-c6bd-4641-8023-4c2f47ea22c4 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.429483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.430149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.430427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.430698 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] PATH_INFO: `/v3/projects/ae6011243d1a475caf5bb329af1cd4c2/users/a97fadd45aa549328c8bbbb538f23822/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.432667 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6011243d1a475caf5bb329af1cd4c2, user_id=a97fadd45aa549328c8bbbb538f23822, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.435290 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.446121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1259/2513] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:28 2022] PUT /identity/v3/projects/ae6011243d1a475caf5bb329af1cd4c2/users/a97fadd45aa549328c8bbbb538f23822/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:28.466882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbdb76-05c7-41a2-bb31-3fdeed301cb3 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.467246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbdb76-05c7-41a2-bb31-3fdeed301cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.467637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbdb76-05c7-41a2-bb31-3fdeed301cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.498871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-55fbdb76-05c7-41a2-bb31-3fdeed301cb3 None None] MFA Rules not processed for user `a97fadd45aa549328c8bbbb538f23822`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:28.500204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-55fbdb76-05c7-41a2-bb31-3fdeed301cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:28.573558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1255/2514] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:28.585825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.587712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:28.651828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af7b9-592f-447a-968d-74c08d9d5ae2 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.655807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.656558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.656870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.657151 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.668418 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.670012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.680692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1260/2515] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:06:28.762671 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbdb76-05c7-41a2-bb31-3fdeed301cb3 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.774555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbdb76-05c7-41a2-bb31-3fdeed301cb3 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.778421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765ea4a-087a-4585-89b5-4df05ce3098c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.779186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765ea4a-087a-4585-89b5-4df05ce3098c service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.779588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765ea4a-087a-4585-89b5-4df05ce3098c service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.780220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765ea4a-087a-4585-89b5-4df05ce3098c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.845619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765ea4a-087a-4585-89b5-4df05ce3098c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.847076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765ea4a-087a-4585-89b5-4df05ce3098c service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.856332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1256/2516] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:38.676500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=41,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:38.852024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=81 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:44.800969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:44.867915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e57dd-b954-4449-b14f-7c5852162ca3 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:44.872838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:44.873469 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:44.873776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:44.874049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] PATH_INFO: `/v3/users/62a5a274301846f8ad502984e8fb826b` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:44.875182 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a5a274301846f8ad502984e8fb826b)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:44.877149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:44.933598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a5a274301846f8ad502984e8fb826b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:44.934356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a5a274301846f8ad502984e8fb826b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:44.943518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a5a274301846f8ad502984e8fb826b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:44.948811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a5a274301846f8ad502984e8fb826b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:44.963896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a5a274301846f8ad502984e8fb826b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:44.982761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1261/2517] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:06:44 2022] DELETE /identity/v3/users/62a5a274301846f8ad502984e8fb826b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:44.987216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:44.987660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:44.987778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:45.007783 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:45.009044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:45.093786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1257/2518] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:45.105555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:45.107000 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:45.163749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02729471-5369-4a11-8daa-cb7cf2a350f0 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:45.167475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:45.168083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:45.168860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:45.168860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:45.178807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:45.180414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:45.191739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1262/2519] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:45.501717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:45.503226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:45.543821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38590-f246-4dce-a5c0-de0ef694553c None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:45.550149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:45.552352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:45.552644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:45.552923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] PATH_INFO: `/v3/projects/c3ff15d6908d401ea09d6db16dbc2847` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:45.554150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ff15d6908d401ea09d6db16dbc2847)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:45.556061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:45.619670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ff15d6908d401ea09d6db16dbc2847', 'request_id': 'req-875b3219-f44a-4ee9-85b0-9346e8b82cf8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.630918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ff15d6908d401ea09d6db16dbc2847', 'request_id': 'req-875b3219-f44a-4ee9-85b0-9346e8b82cf8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.642266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ff15d6908d401ea09d6db16dbc2847', 'request_id': 'req-875b3219-f44a-4ee9-85b0-9346e8b82cf8'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.666636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] The token cache is being invalidate because project c3ff15d6908d401ea09d6db16dbc2847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:06:45.667039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.667743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.669577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1258/2520] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:06:45 2022] DELETE /identity/v3/projects/c3ff15d6908d401ea09d6db16dbc2847 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:45.673356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:45.685812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f1c79-962d-4f3b-9fe7-8d31a14357af service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:45.711984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:45.774985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:45.775549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:45.775831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:45.776108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] PATH_INFO: `/v3/users/a97fadd45aa549328c8bbbb538f23822` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:45.777575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a97fadd45aa549328c8bbbb538f23822)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:45.779504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:45.836920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a97fadd45aa549328c8bbbb538f23822'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.837896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a97fadd45aa549328c8bbbb538f23822'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.845810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a97fadd45aa549328c8bbbb538f23822'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.846735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a97fadd45aa549328c8bbbb538f23822'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.863737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ed8bd65c-f6c8-453b-be1e-d407d6e82fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a97fadd45aa549328c8bbbb538f23822'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:45.880393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1263/2521] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:06:45 2022] DELETE /identity/v3/users/a97fadd45aa549328c8bbbb538f23822 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:06:46.270870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:46.272381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:46.289844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b3219-f44a-4ee9-85b0-9346e8b82cf8 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:46.368476 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:46.369231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.369588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.369937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] PATH_INFO: `/v3/projects/ae6011243d1a475caf5bb329af1cd4c2` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.371240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6011243d1a475caf5bb329af1cd4c2)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:46.373369 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:46.421605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:46.457337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6011243d1a475caf5bb329af1cd4c2', 'request_id': 'req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:46.469447 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6011243d1a475caf5bb329af1cd4c2', 'request_id': 'req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:46.477041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6011243d1a475caf5bb329af1cd4c2', 'request_id': 'req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:46.503934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] The token cache is being invalidate because project ae6011243d1a475caf5bb329af1cd4c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:06:46.504183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:46.506716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:46.511617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1259/2522] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:06:46 2022] DELETE /identity/v3/projects/ae6011243d1a475caf5bb329af1cd4c2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:46.513824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8846ef-30dd-4272-b2e1-09df41c35f6d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.514252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8846ef-30dd-4272-b2e1-09df41c35f6d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.514627 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8846ef-30dd-4272-b2e1-09df41c35f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.540973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-ff8846ef-30dd-4272-b2e1-09df41c35f6d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:46.542854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ff8846ef-30dd-4272-b2e1-09df41c35f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:46.548713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:46.550621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:46.627152 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1264/2523] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:46.635025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096680b-6e7b-407b-a27f-8a1695adda93 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.635914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096680b-6e7b-407b-a27f-8a1695adda93 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.635914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096680b-6e7b-407b-a27f-8a1695adda93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.650895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98cf1a-f681-43c2-8c66-f4e33ba37ae5 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:46.656239 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:46.657570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.658383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.658829 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.659612 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5096680b-6e7b-407b-a27f-8a1695adda93 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:46.661523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5096680b-6e7b-407b-a27f-8a1695adda93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:46.663446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:46.672116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1265/2524] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:06:46.676392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096680b-6e7b-407b-a27f-8a1695adda93 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:46.688965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096680b-6e7b-407b-a27f-8a1695adda93 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:46.695175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:46.695709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.695991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.696389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.697335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:06:46.697643 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:46.699640 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:46.716798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1266/2525] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:06:46.721439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:46.735061 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4ee8-33ce-45ad-99cc-5e690f5fdc7c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:46.741637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:46.743013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.743367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.745064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.745899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:46.748448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:46.750590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:46.752599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:46.768464 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1260/2526] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:06:46.776194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:46.796306 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1267/2527] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:06:46 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:06:46.797930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:46.817638 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790b75a-670e-47a1-a27b-762a4bd86238 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:46.898210 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:46.898821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.899102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.899393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.899990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:46.901984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:46.922771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:46.946585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:46.948575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1261/2528] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:06:46 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:46.958888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd734f-2f94-4cee-95ca-8435fdfc147b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:46.965155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:46.966570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.967046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.967487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.968475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:46.971019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:46.988423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1268/2529] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:46.989710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.001152 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563b79c-5a06-4951-a558-3915214ae6a4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.007239 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5575642-6486-4390-ab5b-ea3247dada52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.007915 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5575642-6486-4390-ab5b-ea3247dada52 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.008228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5575642-6486-4390-ab5b-ea3247dada52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.008730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5575642-6486-4390-ab5b-ea3247dada52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.009404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5575642-6486-4390-ab5b-ea3247dada52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.011047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5575642-6486-4390-ab5b-ea3247dada52 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.027430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1262/2530] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:47.028684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.042319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006951a-03b9-46b8-a2e8-48dc73f51a63 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.048139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.048899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.049314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.049693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] PATH_INFO: `/v3/projects/df5e2f9f35114b48960c15294d47dc69/users/86ab4f0c4d8a47059ab01025444ecd67/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.067010 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5e2f9f35114b48960c15294d47dc69, user_id=86ab4f0c4d8a47059ab01025444ecd67, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.071007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.091031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1269/2531] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:47 2022] PUT /identity/v3/projects/df5e2f9f35114b48960c15294d47dc69/users/86ab4f0c4d8a47059ab01025444ecd67/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:47.102619 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717120-8447-4402-84ea-f77eb5e5f70f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.102909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717120-8447-4402-84ea-f77eb5e5f70f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.103187 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717120-8447-4402-84ea-f77eb5e5f70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.139049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-98717120-8447-4402-84ea-f77eb5e5f70f None None] MFA Rules not processed for user `86ab4f0c4d8a47059ab01025444ecd67`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:47.140495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-98717120-8447-4402-84ea-f77eb5e5f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:47.237045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1263/2532] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:47.242255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.331885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32334f6-d9fb-4ece-81f2-6115d40a7ffc admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.337107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.337765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.338041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.338545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.339198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.341294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.379813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1270/2533] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:06:47 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:06:47.382020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717120-8447-4402-84ea-f77eb5e5f70f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.400110 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717120-8447-4402-84ea-f77eb5e5f70f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.497231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.497860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.498139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.498410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.499047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.501408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.548945 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1264/2534] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:06:47 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:47.550473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.572915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34a8d-0158-435b-8f75-e2df5ca12880 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.581217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.582529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.583116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.583779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.585816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.590652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.612184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1271/2535] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:06:47.616812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.627019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf993c7-3eca-456d-83fc-2262a68cad91 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.631657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.632366 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.632650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.632925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] PATH_INFO: `/v3/projects/f3bc8a352aaa482da020e83e2047a04e/users/706b400f3de64671b64d4a4715617c89/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.648102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bc8a352aaa482da020e83e2047a04e, user_id=706b400f3de64671b64d4a4715617c89, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.650780 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.664505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1265/2536] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:47 2022] PUT /identity/v3/projects/f3bc8a352aaa482da020e83e2047a04e/users/706b400f3de64671b64d4a4715617c89/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:47.669426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.680325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedf890-8c25-4c59-9106-32669f230baa admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.739863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c08c5-e203-4677-bc09-0bd865b1bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.741096 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c08c5-e203-4677-bc09-0bd865b1bdfd admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.741376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c08c5-e203-4677-bc09-0bd865b1bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.741643 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c08c5-e203-4677-bc09-0bd865b1bdfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.742280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c08c5-e203-4677-bc09-0bd865b1bdfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.743857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c08c5-e203-4677-bc09-0bd865b1bdfd admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.753774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1272/2537] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:47.758033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.768494 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4c7a-d79d-4b65-a5ab-cb989e67ba67 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.773025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.773703 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.773975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.774243 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] PATH_INFO: `/v3/projects/f3bc8a352aaa482da020e83e2047a04e/users/706b400f3de64671b64d4a4715617c89/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.776180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bc8a352aaa482da020e83e2047a04e, user_id=706b400f3de64671b64d4a4715617c89, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:47.779090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:47.790033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1266/2538] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:47 2022] PUT /identity/v3/projects/f3bc8a352aaa482da020e83e2047a04e/users/706b400f3de64671b64d4a4715617c89/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:06:47.800772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.801001 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.801272 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:47.829186 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] MFA Rules not processed for user `706b400f3de64671b64d4a4715617c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:47.830398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:47.857679 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:47.909131 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1273/2539] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:47.920640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:47.922551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:47.953203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:47.989885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472077-4925-40af-81c3-dcf718cd0916 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:47.994064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:47.994693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:47.995153 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:47.995432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:48.057694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:48.059252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:48.068534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1267/2540] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:06:48.616197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:48.617632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:48.711318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136bd65-bbd4-47e3-8d22-25abee1c580f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:48.719431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:48.720133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:48.720504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:48.720840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:48.722846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:48.736436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:48.738109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:48.750228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1274/2541] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:54.792277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:54.800967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55e392-b154-4889-b289-0c0c3facab9c service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:54.805157 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:54.805792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:54.806070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:54.806346 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:54.815396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:54.817001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:54.827587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1268/2542] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:55.410530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:55.465722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a5825-0f09-46fc-9ae6-b14146af1264 service nova] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:55.469315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:55.469920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:55.470199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:55.470470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:55.472233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:55.530922 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:55.533080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:55.535055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:55.545570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1275/2543] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:58.681267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:58.683391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:58.701813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b921d-69df-4254-b028-fe54d5218b19 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:58.706596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:58.707256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:58.707512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:58.707785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] PATH_INFO: `/v3/users/7a701149974b45db9f782bc5a232aabe` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:58.708944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a701149974b45db9f782bc5a232aabe)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:58.711623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:58.763914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a701149974b45db9f782bc5a232aabe'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:58.770676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a701149974b45db9f782bc5a232aabe'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:58.779297 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a701149974b45db9f782bc5a232aabe'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:58.780060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a701149974b45db9f782bc5a232aabe'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:58.792886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-15a3ab26-e4fd-46f5-b59c-c58e3fb3a4d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a701149974b45db9f782bc5a232aabe'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:58.798989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1269/2544] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:06:58 2022] DELETE /identity/v3/users/7a701149974b45db9f782bc5a232aabe => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:58.973292 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:58.983652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807d710-f5d3-4ac1-901c-0776da4a6a94 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.041117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.041745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.042020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.042290 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] PATH_INFO: `/v3/projects/a5cd009fd06248eebfe0c84b055ef3e0` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.043394 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5cd009fd06248eebfe0c84b055ef3e0)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.045332 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.091274 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5cd009fd06248eebfe0c84b055ef3e0', 'request_id': 'req-6c022da6-48e9-402b-919a-f3a5fe195fcb'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:59.096625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5cd009fd06248eebfe0c84b055ef3e0', 'request_id': 'req-6c022da6-48e9-402b-919a-f3a5fe195fcb'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:59.103026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5cd009fd06248eebfe0c84b055ef3e0', 'request_id': 'req-6c022da6-48e9-402b-919a-f3a5fe195fcb'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:59.115378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] The token cache is being invalidate because project a5cd009fd06248eebfe0c84b055ef3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:06:59.115609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:59.116762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-6c022da6-48e9-402b-919a-f3a5fe195fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:06:59.118603 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1276/2545] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:06:58 2022] DELETE /identity/v3/projects/a5cd009fd06248eebfe0c84b055ef3e0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:59.123340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4247-2039-4e93-9363-7fc79f3de37a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.123628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4247-2039-4e93-9363-7fc79f3de37a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.123899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4247-2039-4e93-9363-7fc79f3de37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.143970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e2ea4247-2039-4e93-9363-7fc79f3de37a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:59.145198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e2ea4247-2039-4e93-9363-7fc79f3de37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.177836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:59.207414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1270/2546] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:59.215189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eaf47e-6e37-449c-8abe-b31dbe1b094b None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.215534 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eaf47e-6e37-449c-8abe-b31dbe1b094b None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.215815 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eaf47e-6e37-449c-8abe-b31dbe1b094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.234974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-42eaf47e-6e37-449c-8abe-b31dbe1b094b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:59.236227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-42eaf47e-6e37-449c-8abe-b31dbe1b094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.244920 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1277/2547] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:59.248971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea4247-2039-4e93-9363-7fc79f3de37a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.259445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea4247-2039-4e93-9363-7fc79f3de37a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.264186 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.264801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.265080 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.265468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.266116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:06:59.266590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.268262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.288104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1271/2548] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:06:59.290525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eaf47e-6e37-449c-8abe-b31dbe1b094b None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.302099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eaf47e-6e37-449c-8abe-b31dbe1b094b None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.307736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.308517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.308803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.309081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.309751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.311690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.348518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1278/2549] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:06:59 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:59.349870 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.360668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546c6d8-2cc5-4ab0-aac9-b74784150ec7 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.427620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.429517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.429517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.429517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.430353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.432419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.457409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1272/2550] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:06:59 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:06:59.461564 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.474862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81270bc-5c5c-4153-b0bd-39bfae5f4f52 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.480733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.481429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.481713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.481984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.482625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.484266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.495009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1279/2551] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:06:59.499119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.509395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aae429-40c1-49a9-af29-7dd1af0ec57f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.514111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbe7b7-a74c-4958-9e66-6b21a6f703f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.514723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbe7b7-a74c-4958-9e66-6b21a6f703f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.515005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbe7b7-a74c-4958-9e66-6b21a6f703f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.515277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbe7b7-a74c-4958-9e66-6b21a6f703f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.515921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbe7b7-a74c-4958-9e66-6b21a6f703f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.517575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbe7b7-a74c-4958-9e66-6b21a6f703f2 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.529451 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1273/2552] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:06:59.533544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.544670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d746999-3238-44e7-9c2c-f7638c1cfe0c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.549497 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.550163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.550452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.550725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] PATH_INFO: `/v3/projects/942cc6b640f64429aa130658d89f25d6/users/07bcbe8536dd4868ae07dd98f15b3529/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.566764 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942cc6b640f64429aa130658d89f25d6, user_id=07bcbe8536dd4868ae07dd98f15b3529, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.569537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.583567 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1280/2553] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:06:59 2022] PUT /identity/v3/projects/942cc6b640f64429aa130658d89f25d6/users/07bcbe8536dd4868ae07dd98f15b3529/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:06:59.595429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.595825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.596183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.627401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] MFA Rules not processed for user `07bcbe8536dd4868ae07dd98f15b3529`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:59.628708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.694199 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1274/2554] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:06:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:59.705415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.707083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.733605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:59.767241 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37551-f179-4cc3-8664-876b24225564 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.770850 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.771455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.771730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.771996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.780762 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.782256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.791422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1281/2555] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:09.691163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:09.787633 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=16 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:14.733145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:14.734597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:14.791172 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f135d50-770d-428f-b021-125e9c3cf897 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:14.794742 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:14.795358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:14.795636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:14.795904 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:14.797624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:14.855981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:14.857404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:14.866456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1275/2556] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:15.738527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:15.741605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:15.881914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49530024-9dfe-4448-9a1b-2e07389f6700 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:15.895153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693c24-a787-4fd4-ba83-2f642dd9529b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86456c6dbe4e454e9f255960b70c0832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:15.896823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693c24-a787-4fd4-ba83-2f642dd9529b service nova] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:15.897269 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693c24-a787-4fd4-ba83-2f642dd9529b service nova] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:15.897595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693c24-a787-4fd4-ba83-2f642dd9529b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:15.899975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0d693c24-a787-4fd4-ba83-2f642dd9529b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:15.919321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693c24-a787-4fd4-ba83-2f642dd9529b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:15.921719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693c24-a787-4fd4-ba83-2f642dd9529b service nova] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:15.939150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1282/2557] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:16.543776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:16.610078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065e938-83a0-49fe-ae35-eb35ee2fdc39 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:16.614699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:16.615327 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:16.615643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:16.615920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] PATH_INFO: `/v3/users/07bcbe8536dd4868ae07dd98f15b3529` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:16.617130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bcbe8536dd4868ae07dd98f15b3529)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:16.620040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:16.670549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bcbe8536dd4868ae07dd98f15b3529'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:16.677315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bcbe8536dd4868ae07dd98f15b3529'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:16.678981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:16.686776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bcbe8536dd4868ae07dd98f15b3529'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:16.687803 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bcbe8536dd4868ae07dd98f15b3529'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:16.701840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bcbe8536dd4868ae07dd98f15b3529'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:16.708173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1276/2558] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:16 2022] DELETE /identity/v3/users/07bcbe8536dd4868ae07dd98f15b3529 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:16.713350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:16.713647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:16.713982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:16.734864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:16.736085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:16.796593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1283/2559] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:16.806146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:16.808379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:16.864597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd6fdf-167b-4cfd-bb72-a7fe2ee19b96 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:16.868198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0cd9d-8498-4c0b-8098-18cbef4ee554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:16.868798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0cd9d-8498-4c0b-8098-18cbef4ee554 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:16.869760 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0cd9d-8498-4c0b-8098-18cbef4ee554 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:16.870036 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0cd9d-8498-4c0b-8098-18cbef4ee554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:16.878819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0cd9d-8498-4c0b-8098-18cbef4ee554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:16.880279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0cd9d-8498-4c0b-8098-18cbef4ee554 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:16.889701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1277/2560] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:17.156553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.158193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:17.171397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9256-7ae4-4f80-a9dd-3f791936311f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.176086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.176777 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.177053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.177326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] PATH_INFO: `/v3/projects/942cc6b640f64429aa130658d89f25d6` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.178449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942cc6b640f64429aa130658d89f25d6)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.180555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.231218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942cc6b640f64429aa130658d89f25d6', 'request_id': 'req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:17.237172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942cc6b640f64429aa130658d89f25d6', 'request_id': 'req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:17.243740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942cc6b640f64429aa130658d89f25d6', 'request_id': 'req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:17.259765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] The token cache is being invalidate because project 942cc6b640f64429aa130658d89f25d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:17.259765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:17.259765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-af089a3b-a8c6-4ec7-9a5a-ee1a7d06cb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:17.259765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1284/2561] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:17 2022] DELETE /identity/v3/projects/942cc6b640f64429aa130658d89f25d6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:17.264214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5dfe6-2700-464d-ab94-3d0095f29813 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.264507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5dfe6-2700-464d-ab94-3d0095f29813 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.264785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5dfe6-2700-464d-ab94-3d0095f29813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.284330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a0d5dfe6-2700-464d-ab94-3d0095f29813 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:17.285536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a0d5dfe6-2700-464d-ab94-3d0095f29813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:17.345008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1278/2562] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:17.352771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5776975-250e-46cc-a50a-e29dc641f8f5 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.353062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5776975-250e-46cc-a50a-e29dc641f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.353956 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5776975-250e-46cc-a50a-e29dc641f8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.373014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e5776975-250e-46cc-a50a-e29dc641f8f5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:17.374200 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e5776975-250e-46cc-a50a-e29dc641f8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:17.381953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1285/2563] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:17.386533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5dfe6-2700-464d-ab94-3d0095f29813 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.396061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5dfe6-2700-464d-ab94-3d0095f29813 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.401278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.401867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.402146 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.402419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.403057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:17.403361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.404985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.416566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1279/2564] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:07:17.421063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5776975-250e-46cc-a50a-e29dc641f8f5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.425767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:17.432932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5776975-250e-46cc-a50a-e29dc641f8f5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.439100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.439715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.439996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.440308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.440948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.442840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.472557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1286/2565] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:17 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:17.476801 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.486719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1e575-a0ae-4344-a51a-9cfeb8c94d15 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.542509 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.543019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.543298 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.543568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.544218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.546139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.555704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:17.569317 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1280/2566] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:17 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:17.573923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.584103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5040c-9955-4666-a268-6e15e33dc7d6 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.589586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.590177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.590451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.590745 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.591374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.593003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.602619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1287/2567] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:17.606545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.616536 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96268a-37ba-4188-b2f1-326d53d19a76 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.621295 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa539bf-790a-473a-8073-c43b7adce4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.621889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa539bf-790a-473a-8073-c43b7adce4de admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.622169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa539bf-790a-473a-8073-c43b7adce4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.622586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa539bf-790a-473a-8073-c43b7adce4de admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.623074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa539bf-790a-473a-8073-c43b7adce4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.624713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa539bf-790a-473a-8073-c43b7adce4de admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.635729 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1281/2568] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:17.639742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.649722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ec49-ab2d-45a5-a399-24d95c7b5bf8 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.655016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.655663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.655944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.656261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] PATH_INFO: `/v3/projects/a38760d51ff8460eaefd9c9a53a929ee/users/f11c66c2a2624136ad758638616082fe/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.670502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38760d51ff8460eaefd9c9a53a929ee, user_id=f11c66c2a2624136ad758638616082fe, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.673206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.685969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1288/2569] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:17 2022] PUT /identity/v3/projects/a38760d51ff8460eaefd9c9a53a929ee/users/f11c66c2a2624136ad758638616082fe/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:17.696171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b58c56-e0b7-41df-8dc5-0e4f7a2229fa None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.696456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b58c56-e0b7-41df-8dc5-0e4f7a2229fa None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.696795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b58c56-e0b7-41df-8dc5-0e4f7a2229fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.727677 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-30b58c56-e0b7-41df-8dc5-0e4f7a2229fa None None] MFA Rules not processed for user `f11c66c2a2624136ad758638616082fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:17.728953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-30b58c56-e0b7-41df-8dc5-0e4f7a2229fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:17.796007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1282/2570] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:17.800734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.861731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832bac9-6ced-4e02-9132-4b4d8e2fdb17 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.866378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.866976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.867253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.867519 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.868225 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.870796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.897055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1289/2571] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:17 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:17.901325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b58c56-e0b7-41df-8dc5-0e4f7a2229fa None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.912283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b58c56-e0b7-41df-8dc5-0e4f7a2229fa None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.971566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.972206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.972489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.972763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.973387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.975299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.996645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1283/2572] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:17 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:18.000810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:18.012012 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7dfcf-c528-452e-a863-10e33278f211 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:18.016645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:18.017266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.017556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.017794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.018427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:18.020016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:18.031009 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1290/2573] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:18.035085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:18.046001 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419a4c0-3fc5-489c-8409-c547c0df97c1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:18.050693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:18.051287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.051569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.051843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] PATH_INFO: `/v3/projects/00413c1f50d54b2cadfd8b514d9f4950/users/246dded8af304a43bdb1029b9d1ee7ed/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.067653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00413c1f50d54b2cadfd8b514d9f4950, user_id=246dded8af304a43bdb1029b9d1ee7ed, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:18.070359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:18.083926 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1284/2574] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:18 2022] PUT /identity/v3/projects/00413c1f50d54b2cadfd8b514d9f4950/users/246dded8af304a43bdb1029b9d1ee7ed/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:18.088839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:18.098638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c4f9c-f9fa-4378-8039-a7e8db9969e6 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:18.155885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f3e40-6273-4c72-8707-61d2f0b1eeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:18.156529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f3e40-6273-4c72-8707-61d2f0b1eeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.156812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f3e40-6273-4c72-8707-61d2f0b1eeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.157088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f3e40-6273-4c72-8707-61d2f0b1eeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.157724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f3e40-6273-4c72-8707-61d2f0b1eeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:18.159349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f3e40-6273-4c72-8707-61d2f0b1eeb3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:18.168861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1291/2575] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:18.173010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:18.183190 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e6cd8-0d3a-4cc0-bf29-7e5324ba5066 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:18.187740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:18.188414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.188691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.188957 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] PATH_INFO: `/v3/projects/00413c1f50d54b2cadfd8b514d9f4950/users/246dded8af304a43bdb1029b9d1ee7ed/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.190836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00413c1f50d54b2cadfd8b514d9f4950, user_id=246dded8af304a43bdb1029b9d1ee7ed, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:18.193484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:18.203777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1285/2576] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:18 2022] PUT /identity/v3/projects/00413c1f50d54b2cadfd8b514d9f4950/users/246dded8af304a43bdb1029b9d1ee7ed/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:18.216071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1b8aa-8238-4d78-a8c8-1521f91a67d3 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.216396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1b8aa-8238-4d78-a8c8-1521f91a67d3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.216676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1b8aa-8238-4d78-a8c8-1521f91a67d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.244593 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-99e1b8aa-8238-4d78-a8c8-1521f91a67d3 None None] MFA Rules not processed for user `246dded8af304a43bdb1029b9d1ee7ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:18.245838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-99e1b8aa-8238-4d78-a8c8-1521f91a67d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:18.250184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:18.317637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1292/2577] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:18.329035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:18.330954 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:18.360232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:18.392471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daade31a-eac8-4a88-988e-2308ceaf7897 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:18.396086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ebed6-97a5-4a98-8933-574ab91f696a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:18.396736 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ebed6-97a5-4a98-8933-574ab91f696a service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.397012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ebed6-97a5-4a98-8933-574ab91f696a service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.397284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ebed6-97a5-4a98-8933-574ab91f696a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.407100 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ebed6-97a5-4a98-8933-574ab91f696a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:18.409000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ebed6-97a5-4a98-8933-574ab91f696a service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:18.419806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1286/2578] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:20.515201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1b8aa-8238-4d78-a8c8-1521f91a67d3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:20.578367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1b8aa-8238-4d78-a8c8-1521f91a67d3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:20.582950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:20.583584 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:20.583868 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:20.584162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] PATH_INFO: `/v3/users/f11c66c2a2624136ad758638616082fe` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:20.585325 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11c66c2a2624136ad758638616082fe)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:20.588179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:20.636300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11c66c2a2624136ad758638616082fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:20.638289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11c66c2a2624136ad758638616082fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:20.646014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11c66c2a2624136ad758638616082fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:20.647565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11c66c2a2624136ad758638616082fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:20.664785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11c66c2a2624136ad758638616082fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:20.677825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1293/2579] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:20 2022] DELETE /identity/v3/users/f11c66c2a2624136ad758638616082fe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:20.682371 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:20.682884 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:20.682988 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:20.702091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:20.703302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:20.766984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1287/2580] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:20.777405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:20.778838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:20.802003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:20.842947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79158f4-4c79-4c74-a53a-dc84f075b35b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:20.846715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:20.847341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:20.847685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:20.847987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:20.858479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:20.860247 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:20.871559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1294/2581] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:21.168028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:21.169668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:21.188060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d4ca9-a33b-4ca6-9b4b-5cdeb474f5e0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:21.193106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:21.193931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:21.194209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:21.194542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] PATH_INFO: `/v3/projects/a38760d51ff8460eaefd9c9a53a929ee` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:21.195669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38760d51ff8460eaefd9c9a53a929ee)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:21.197782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:21.245395 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38760d51ff8460eaefd9c9a53a929ee', 'request_id': 'req-9bff8ffe-3a36-40e5-a897-88b1abf74219'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.251359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38760d51ff8460eaefd9c9a53a929ee', 'request_id': 'req-9bff8ffe-3a36-40e5-a897-88b1abf74219'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.257896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38760d51ff8460eaefd9c9a53a929ee', 'request_id': 'req-9bff8ffe-3a36-40e5-a897-88b1abf74219'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.270688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] The token cache is being invalidate because project a38760d51ff8460eaefd9c9a53a929ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:21.270975 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.271592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.273150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1288/2582] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:21 2022] DELETE /identity/v3/projects/a38760d51ff8460eaefd9c9a53a929ee => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:21.277393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:21.287352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985132ba-28d6-466f-a0b6-c6e1a279d7f8 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:21.346215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:21.346846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:21.347122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:21.347389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] PATH_INFO: `/v3/users/246dded8af304a43bdb1029b9d1ee7ed` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:21.348526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246dded8af304a43bdb1029b9d1ee7ed)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:21.350432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:21.395356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246dded8af304a43bdb1029b9d1ee7ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.396007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246dded8af304a43bdb1029b9d1ee7ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.403156 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246dded8af304a43bdb1029b9d1ee7ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.403771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246dded8af304a43bdb1029b9d1ee7ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.416227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-c9f28500-06b4-4187-a19c-878a8ef8a729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246dded8af304a43bdb1029b9d1ee7ed'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.432881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1295/2583] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:21 2022] DELETE /identity/v3/users/246dded8af304a43bdb1029b9d1ee7ed => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:21.711178 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:21.713302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:21.731841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff8ffe-3a36-40e5-a897-88b1abf74219 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:21.779850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:21.791182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:21.791804 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:21.792081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:21.792385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] PATH_INFO: `/v3/projects/00413c1f50d54b2cadfd8b514d9f4950` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:21.793525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00413c1f50d54b2cadfd8b514d9f4950)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:21.795418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:21.840809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00413c1f50d54b2cadfd8b514d9f4950', 'request_id': 'req-505853a2-32c9-4d86-8a7b-479447984513'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.847097 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00413c1f50d54b2cadfd8b514d9f4950', 'request_id': 'req-505853a2-32c9-4d86-8a7b-479447984513'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.853534 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00413c1f50d54b2cadfd8b514d9f4950', 'request_id': 'req-505853a2-32c9-4d86-8a7b-479447984513'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.866508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] The token cache is being invalidate because project 00413c1f50d54b2cadfd8b514d9f4950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:21.866834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.867455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-505853a2-32c9-4d86-8a7b-479447984513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:21.869592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1289/2584] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:21 2022] DELETE /identity/v3/projects/00413c1f50d54b2cadfd8b514d9f4950 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:21.875097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340154a-252d-4b78-a65d-187a8fb78ef3 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:21.875594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340154a-252d-4b78-a65d-187a8fb78ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:21.875772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340154a-252d-4b78-a65d-187a8fb78ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:21.896082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f340154a-252d-4b78-a65d-187a8fb78ef3 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:21.897385 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f340154a-252d-4b78-a65d-187a8fb78ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:21.936024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:21.959599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1296/2585] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:21.968678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574f71-711a-435f-ac4d-c5a759738968 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:21.968968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574f71-711a-435f-ac4d-c5a759738968 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:21.969246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574f71-711a-435f-ac4d-c5a759738968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:21.990552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f5574f71-711a-435f-ac4d-c5a759738968 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:21.991762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f5574f71-711a-435f-ac4d-c5a759738968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:21.999796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1290/2586] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:22.004192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340154a-252d-4b78-a65d-187a8fb78ef3 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.015697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340154a-252d-4b78-a65d-187a8fb78ef3 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.020273 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.020872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.021149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.021415 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.022056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:22.022359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.023958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.037651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1297/2587] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:07:22.042115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574f71-711a-435f-ac4d-c5a759738968 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.051437 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574f71-711a-435f-ac4d-c5a759738968 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.056122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.056755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.057035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.057307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.057940 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.059844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.088495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1291/2588] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:22 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:22.092772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.104022 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda709c-92d5-420b-95a5-4b2d32e0497d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.175183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.176072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.176442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.176776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.177669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.179675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.210349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1298/2589] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:22 2022] POST /identity/v3/users => generated 301 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:22.213728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.229135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecc8b-7355-43d2-a199-82f4505d6c99 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.241746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.242573 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.242910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.243230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.244566 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.246208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.258506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1292/2590] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:22.262602 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.277380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2bdb9-cbd3-4b4e-814b-545c1a1e23db admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.285700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8ef79-9c64-4b7f-bad1-3d3837d9dad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.287034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8ef79-9c64-4b7f-bad1-3d3837d9dad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.287034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8ef79-9c64-4b7f-bad1-3d3837d9dad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.287298 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8ef79-9c64-4b7f-bad1-3d3837d9dad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.288672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8ef79-9c64-4b7f-bad1-3d3837d9dad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.290620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8ef79-9c64-4b7f-bad1-3d3837d9dad6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.304157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1299/2591] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:22.308929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.327023 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dd018-02bc-49ac-94fe-af60b91081e9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.331898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.332648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.332933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.333383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] PATH_INFO: `/v3/projects/06391e12ac924ed38eb09b844ee5c668/users/bd7fb27c2e4f4c5f9ff723ebf6cac972/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.352195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06391e12ac924ed38eb09b844ee5c668, user_id=bd7fb27c2e4f4c5f9ff723ebf6cac972, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.355713 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.372473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1293/2592] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:22 2022] PUT /identity/v3/projects/06391e12ac924ed38eb09b844ee5c668/users/bd7fb27c2e4f4c5f9ff723ebf6cac972/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:22.383239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b713d-9036-4c7f-b6fc-bd3cd7cdc823 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.383239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b713d-9036-4c7f-b6fc-bd3cd7cdc823 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.383810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b713d-9036-4c7f-b6fc-bd3cd7cdc823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.422914 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-e89b713d-9036-4c7f-b6fc-bd3cd7cdc823 None None] MFA Rules not processed for user `bd7fb27c2e4f4c5f9ff723ebf6cac972`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:22.423230 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-e89b713d-9036-4c7f-b6fc-bd3cd7cdc823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:22.531686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1300/2593] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:22.536676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.612385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d5839-11f7-4ba7-97c6-80432a954814 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.617106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.617718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.617996 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.618272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.618918 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.620912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.653056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1294/2594] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:22 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:22.657128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b713d-9036-4c7f-b6fc-bd3cd7cdc823 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.668175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b713d-9036-4c7f-b6fc-bd3cd7cdc823 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.701299 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:22.724784 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.725410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.725719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.725991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.726610 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.728578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.748716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1301/2595] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:22 2022] POST /identity/v3/users => generated 306 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:22.752768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.758449 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:22.765150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840a52-c243-4d48-be54-c2ec8bf8455f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.770796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.771383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.771662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.771934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.772616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.774275 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.785974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1295/2596] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:22.790188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.800439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bcf55-87c6-43c4-909e-557549c8704e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.804937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.805609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.805887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.806165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] PATH_INFO: `/v3/projects/4646a93fbfe34bd1a58e6307e6e1bdcd/users/ec4353bbce254e28ad1373e80bf6c26e/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.822333 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4646a93fbfe34bd1a58e6307e6e1bdcd, user_id=ec4353bbce254e28ad1373e80bf6c26e, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.825068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.839296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1302/2597] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:22 2022] PUT /identity/v3/projects/4646a93fbfe34bd1a58e6307e6e1bdcd/users/ec4353bbce254e28ad1373e80bf6c26e/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:22.843250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.853854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab632a5-cbbe-4eb4-9049-78a101cc1d59 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.920319 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7350b-62e6-45ca-a296-49cdf4279b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.920950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7350b-62e6-45ca-a296-49cdf4279b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.921231 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7350b-62e6-45ca-a296-49cdf4279b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.921711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7350b-62e6-45ca-a296-49cdf4279b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.922374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7350b-62e6-45ca-a296-49cdf4279b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.924329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7350b-62e6-45ca-a296-49cdf4279b05 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.936805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1296/2598] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:22.941149 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.953317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f25769-0e7d-48ed-8560-f0b103d824f9 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:22.958722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:22.959398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.959742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.960240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] PATH_INFO: `/v3/projects/4646a93fbfe34bd1a58e6307e6e1bdcd/users/ec4353bbce254e28ad1373e80bf6c26e/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.962573 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4646a93fbfe34bd1a58e6307e6e1bdcd, user_id=ec4353bbce254e28ad1373e80bf6c26e, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:22.965315 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:22.981458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1303/2599] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:22 2022] PUT /identity/v3/projects/4646a93fbfe34bd1a58e6307e6e1bdcd/users/ec4353bbce254e28ad1373e80bf6c26e/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:22.993303 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcbbb-bb31-410d-9c30-3f9ea8ffd82a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.993652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcbbb-bb31-410d-9c30-3f9ea8ffd82a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.994752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcbbb-bb31-410d-9c30-3f9ea8ffd82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:23.026388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3f2bcbbb-bb31-410d-9c30-3f9ea8ffd82a None None] MFA Rules not processed for user `ec4353bbce254e28ad1373e80bf6c26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:23.027620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3f2bcbbb-bb31-410d-9c30-3f9ea8ffd82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:23.105354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1297/2600] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:23.114599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:23.116733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:23.173900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7ae25-7a79-476f-a44d-defdf10dc945 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:23.177585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:23.178185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:23.178464 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:23.178733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:23.187535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:23.188978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:23.198197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1304/2601] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:23.277620 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcbbb-bb31-410d-9c30-3f9ea8ffd82a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:23.287274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcbbb-bb31-410d-9c30-3f9ea8ffd82a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:23.291504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:23.292096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:23.292409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:23.292681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:23.350304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:23.351773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:23.362659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1298/2602] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:23 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:24.713966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:24.715629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:24.778005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201d97b-ba05-4a4d-b56e-eee040d51664 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:24.781840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181afcd-1ad6-42ae-a9fb-6087d2f06689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:24.782343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181afcd-1ad6-42ae-a9fb-6087d2f06689 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:24.782620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181afcd-1ad6-42ae-a9fb-6087d2f06689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:24.782889 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181afcd-1ad6-42ae-a9fb-6087d2f06689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:24.784622 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1181afcd-1ad6-42ae-a9fb-6087d2f06689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:24.832191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:24.851788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181afcd-1ad6-42ae-a9fb-6087d2f06689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:24.853348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181afcd-1ad6-42ae-a9fb-6087d2f06689 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:24.862887 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1305/2603] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:26.141268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:26.142750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:26.157016 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214fddd-7c2a-4231-94a3-af25479b04b8 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:26.163044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:26.163678 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:26.163955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:26.164337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] PATH_INFO: `/v3/users/86ab4f0c4d8a47059ab01025444ecd67` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:26.165488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ab4f0c4d8a47059ab01025444ecd67)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:26.167572 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:26.174148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:26.223419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ab4f0c4d8a47059ab01025444ecd67'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.233120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ab4f0c4d8a47059ab01025444ecd67'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.244237 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ab4f0c4d8a47059ab01025444ecd67'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.244920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ab4f0c4d8a47059ab01025444ecd67'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.259415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ab4f0c4d8a47059ab01025444ecd67'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.268839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1299/2604] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:26 2022] DELETE /identity/v3/users/86ab4f0c4d8a47059ab01025444ecd67 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:26.270505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:26.270959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:26.271043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:26.291576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:26.293369 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:26.373305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1306/2605] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:26.384057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:26.387118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:26.459278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34feb9-6103-4596-9661-52a2381e859a admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:26.464471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:26.465542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:26.465542 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:26.465731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:26.475721 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:26.477330 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:26.492642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1300/2606] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:26.794140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:26.796085 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:26.816578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bb9ef-4678-4b6e-ae22-23bb757a1af7 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:26.826310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:26.827457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:26.827928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:26.828254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] PATH_INFO: `/v3/projects/df5e2f9f35114b48960c15294d47dc69` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:26.830392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5e2f9f35114b48960c15294d47dc69)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:26.832674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:26.890429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5e2f9f35114b48960c15294d47dc69', 'request_id': 'req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.898283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5e2f9f35114b48960c15294d47dc69', 'request_id': 'req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.904930 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5e2f9f35114b48960c15294d47dc69', 'request_id': 'req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.922366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] The token cache is being invalidate because project df5e2f9f35114b48960c15294d47dc69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:26.923040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.923875 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:26.925738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1307/2607] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:26 2022] DELETE /identity/v3/projects/df5e2f9f35114b48960c15294d47dc69 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:26.929718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:26.942393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b10459-a62c-4f0a-87e5-dd9964fbdb84 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:27.017663 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:27.018388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:27.018817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:27.019122 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] PATH_INFO: `/v3/users/706b400f3de64671b64d4a4715617c89` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:27.020360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=706b400f3de64671b64d4a4715617c89)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:27.022325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:27.053081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:27.073252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706b400f3de64671b64d4a4715617c89'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.082417 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706b400f3de64671b64d4a4715617c89'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.090659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706b400f3de64671b64d4a4715617c89'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.091332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706b400f3de64671b64d4a4715617c89'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.109933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706b400f3de64671b64d4a4715617c89'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.116086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1301/2608] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:26 2022] DELETE /identity/v3/users/706b400f3de64671b64d4a4715617c89 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:27.510237 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:27.511825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:27.527960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7edd7-9982-4b2e-86cc-83701fdbfe77 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:27.613103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:27.614134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:27.614642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:27.615150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] PATH_INFO: `/v3/projects/f3bc8a352aaa482da020e83e2047a04e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:27.616424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3bc8a352aaa482da020e83e2047a04e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:27.620326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:27.634048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:27.694993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3bc8a352aaa482da020e83e2047a04e', 'request_id': 'req-fa6f7d32-fb0e-48e9-a544-461272f33302'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.702498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3bc8a352aaa482da020e83e2047a04e', 'request_id': 'req-fa6f7d32-fb0e-48e9-a544-461272f33302'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.711575 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3bc8a352aaa482da020e83e2047a04e', 'request_id': 'req-fa6f7d32-fb0e-48e9-a544-461272f33302'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.713688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:27.716435 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:27.726819 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] The token cache is being invalidate because project f3bc8a352aaa482da020e83e2047a04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:27.727268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.728212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-fa6f7d32-fb0e-48e9-a544-461272f33302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.729647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017597aa-440e-4aae-adff-70f5695e2c63 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:27.732334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1308/2609] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:27 2022] DELETE /identity/v3/projects/f3bc8a352aaa482da020e83e2047a04e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:27.734662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d958369-07f4-4928-8adb-669fd3c0f624 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:27.735126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d958369-07f4-4928-8adb-669fd3c0f624 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:27.735457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d958369-07f4-4928-8adb-669fd3c0f624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:27.764617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4d958369-07f4-4928-8adb-669fd3c0f624 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:27.766445 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4d958369-07f4-4928-8adb-669fd3c0f624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:27.807305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:27.808026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:27.808622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:27.808902 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] PATH_INFO: `/v3/users/bd7fb27c2e4f4c5f9ff723ebf6cac972` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:27.810616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7fb27c2e4f4c5f9ff723ebf6cac972)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:27.813577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:27.847441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1309/2610] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:27.856718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3322a-f14e-4d37-97a7-1872e10a43ab None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:27.857159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3322a-f14e-4d37-97a7-1872e10a43ab None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:27.857540 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3322a-f14e-4d37-97a7-1872e10a43ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:27.893468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0fe3322a-f14e-4d37-97a7-1872e10a43ab None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:27.897353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0fe3322a-f14e-4d37-97a7-1872e10a43ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:27.919401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7fb27c2e4f4c5f9ff723ebf6cac972'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.930039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7fb27c2e4f4c5f9ff723ebf6cac972'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.930219 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1310/2611] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:27.937478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3322a-f14e-4d37-97a7-1872e10a43ab None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:27.944839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7fb27c2e4f4c5f9ff723ebf6cac972'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.946064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7fb27c2e4f4c5f9ff723ebf6cac972'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.952106 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3322a-f14e-4d37-97a7-1872e10a43ab None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:27.958356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:27.958975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:27.959254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:27.959524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:27.960235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:27.960793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:27.962718 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:27.972472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b21103eb-c820-4e15-a6b2-9dfb089f2701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7fb27c2e4f4c5f9ff723ebf6cac972'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:27.977367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1311/2612] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:07:27.982802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:27.985413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1302/2613] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:27 2022] DELETE /identity/v3/users/bd7fb27c2e4f4c5f9ff723ebf6cac972 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:27.985944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:27.991560 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:27.991983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:27.992440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.012691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c1ec2-1f88-412e-9b71-11c4f9c663b1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.021407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:28.022689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:28.058070 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:28.120678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.121686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.122167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.122636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.123282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1303/2614] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:28.123734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.126696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.134106 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.135798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:28.171644 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1312/2615] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:27 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:28.175761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.181938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:28.220824 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5e367-6103-455c-8dd6-6deb4eba1d24 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.269173 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f98a0e-10d4-4e47-9db3-11d0dab5a4cf admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.277919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.278548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.278830 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.279111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.279971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.282162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.298311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.298980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.299331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.301171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.307539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1313/2616] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:28 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:28.311945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.318669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.320120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.322356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d34d2-dfd4-4c7a-8103-e742dfa70c0a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.327179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.327790 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.328075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.328400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.329054 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.330474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1304/2617] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:28.336696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.352019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1314/2618] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:28.357401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.375214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb08cb-8e5c-4f3d-8b6e-93da0fad63b5 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.382161 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf93d6a-d49e-4820-9cb8-1f79dc71056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.383226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf93d6a-d49e-4820-9cb8-1f79dc71056e admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.383520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf93d6a-d49e-4820-9cb8-1f79dc71056e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.383797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf93d6a-d49e-4820-9cb8-1f79dc71056e admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.384574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf93d6a-d49e-4820-9cb8-1f79dc71056e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.387234 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf93d6a-d49e-4820-9cb8-1f79dc71056e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.415056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1305/2619] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:28.418224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.439276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765aa2e-f473-4ead-80ab-2b12851af3b6 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.444663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.445495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.445911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.446289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] PATH_INFO: `/v3/projects/2152abdf57024f25a3c91e784205c431/users/f89d855ef443436f85359651a01972af/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.466950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2152abdf57024f25a3c91e784205c431, user_id=f89d855ef443436f85359651a01972af, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.470157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.486210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1315/2620] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:28 2022] PUT /identity/v3/projects/2152abdf57024f25a3c91e784205c431/users/f89d855ef443436f85359651a01972af/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:28.496541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322b328-06ea-4851-a9a4-a18fbfc8602e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.496972 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322b328-06ea-4851-a9a4-a18fbfc8602e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.497353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322b328-06ea-4851-a9a4-a18fbfc8602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.532651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a322b328-06ea-4851-a9a4-a18fbfc8602e None None] MFA Rules not processed for user `f89d855ef443436f85359651a01972af`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:28.534349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a322b328-06ea-4851-a9a4-a18fbfc8602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:28.602614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1306/2621] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:28.616111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.618075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:28.676423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf8035-9536-4059-84a4-99bb7a8a47c3 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.680088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.680730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.681049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.681320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.690493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.691869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.701428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1316/2622] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:28.721768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322b328-06ea-4851-a9a4-a18fbfc8602e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.726107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:28.792784 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322b328-06ea-4851-a9a4-a18fbfc8602e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.797740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.798385 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.798712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.798961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] PATH_INFO: `/v3/users/f89d855ef443436f85359651a01972af` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.800425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f89d855ef443436f85359651a01972af)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.802533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.820191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.822525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:28.842275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6cdbf-c119-48d4-add2-30adb7aff7ef service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:28.847282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:28.848031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.848425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.849211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] PATH_INFO: `/v3/projects/06391e12ac924ed38eb09b844ee5c668` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.850059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=06391e12ac924ed38eb09b844ee5c668)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:28.852048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:28.856098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f89d855ef443436f85359651a01972af'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.866629 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f89d855ef443436f85359651a01972af'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.878144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f89d855ef443436f85359651a01972af'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.878919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f89d855ef443436f85359651a01972af'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.895603 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-99de6a2c-4a48-40a2-ba75-8da4b44e54c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f89d855ef443436f85359651a01972af'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.903549 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1307/2623] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:28 2022] DELETE /identity/v3/users/f89d855ef443436f85359651a01972af => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:28.907967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:28.908287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:28.908565 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:28.914013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06391e12ac924ed38eb09b844ee5c668', 'request_id': 'req-533db8ef-69b7-4e85-a15a-4d3425139f4c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.919513 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:28.922344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06391e12ac924ed38eb09b844ee5c668', 'request_id': 'req-533db8ef-69b7-4e85-a15a-4d3425139f4c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.928111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:28.929689 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:28.932510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06391e12ac924ed38eb09b844ee5c668', 'request_id': 'req-533db8ef-69b7-4e85-a15a-4d3425139f4c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.952191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] The token cache is being invalidate because project 06391e12ac924ed38eb09b844ee5c668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:28.952505 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.953123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:28.957708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1317/2624] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:28 2022] DELETE /identity/v3/projects/06391e12ac924ed38eb09b844ee5c668 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:28.958697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:28.960205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:28.999392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1308/2625] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:29.009750 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:29.011289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:29.034187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533db8ef-69b7-4e85-a15a-4d3425139f4c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:29.039216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:29.039875 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:29.040632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:29.040915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:29.041193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] PATH_INFO: `/v3/users/ec4353bbce254e28ad1373e80bf6c26e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:29.042976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4353bbce254e28ad1373e80bf6c26e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:29.045243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:29.087125 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ab060-b746-4c95-9c44-566d08cbd5fa None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:29.092078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:29.093195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:29.093649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:29.094085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:29.104958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4353bbce254e28ad1373e80bf6c26e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:29.105665 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4353bbce254e28ad1373e80bf6c26e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:29.107349 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:29.109043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:29.114317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4353bbce254e28ad1373e80bf6c26e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:29.114998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4353bbce254e28ad1373e80bf6c26e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:29.122794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1309/2626] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:29.133571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4353bbce254e28ad1373e80bf6c26e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:29.152128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1318/2627] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:28 2022] DELETE /identity/v3/users/ec4353bbce254e28ad1373e80bf6c26e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:29.899037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:29.902250 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:29.939010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c2c0e-60a1-45d7-9ea9-514b1823f982 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:29.943087 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:29.944982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.023991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.024718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.025616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.026482 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] PATH_INFO: `/v3/projects/2152abdf57024f25a3c91e784205c431` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.028000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2152abdf57024f25a3c91e784205c431)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.030229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.052616 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95e010-84d5-4f78-b6bf-46fc39760b4e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.057947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.058660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.058946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.059362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] PATH_INFO: `/v3/projects/4646a93fbfe34bd1a58e6307e6e1bdcd` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.060583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4646a93fbfe34bd1a58e6307e6e1bdcd)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.062639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.065448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:30.129604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2152abdf57024f25a3c91e784205c431', 'request_id': 'req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.146419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2152abdf57024f25a3c91e784205c431', 'request_id': 'req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.146958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4646a93fbfe34bd1a58e6307e6e1bdcd', 'request_id': 'req-0cbf9039-a44d-46eb-8798-a8903c539759'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.155092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4646a93fbfe34bd1a58e6307e6e1bdcd', 'request_id': 'req-0cbf9039-a44d-46eb-8798-a8903c539759'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.159409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2152abdf57024f25a3c91e784205c431', 'request_id': 'req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.163996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4646a93fbfe34bd1a58e6307e6e1bdcd', 'request_id': 'req-0cbf9039-a44d-46eb-8798-a8903c539759'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.178155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] The token cache is being invalidate because project 2152abdf57024f25a3c91e784205c431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:30.178597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.179669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-4ec3dcbe-1a10-4eb3-a6bf-489fded2d9dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.180470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] The token cache is being invalidate because project 4646a93fbfe34bd1a58e6307e6e1bdcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:30.180927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.182474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1310/2628] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:29 2022] DELETE /identity/v3/projects/2152abdf57024f25a3c91e784205c431 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:30.182700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-0cbf9039-a44d-46eb-8798-a8903c539759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:30.189038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1319/2629] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:29 2022] DELETE /identity/v3/projects/4646a93fbfe34bd1a58e6307e6e1bdcd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:30.189217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348402ff-2fce-4e93-a4c4-901094c628e8 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.189217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348402ff-2fce-4e93-a4c4-901094c628e8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.189217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348402ff-2fce-4e93-a4c4-901094c628e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.191165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaf8b0-5ae5-4fda-aae2-a4150b466886 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.191458 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaf8b0-5ae5-4fda-aae2-a4150b466886 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.191734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaf8b0-5ae5-4fda-aae2-a4150b466886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.209501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-348402ff-2fce-4e93-a4c4-901094c628e8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:30.211356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-348402ff-2fce-4e93-a4c4-901094c628e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.213900 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-85eaf8b0-5ae5-4fda-aae2-a4150b466886 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:30.215094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-85eaf8b0-5ae5-4fda-aae2-a4150b466886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.282513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:30.301310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1311/2630] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:30.308579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d548ee-1148-4823-8f73-e9c95d4c73ca None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.309249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d548ee-1148-4823-8f73-e9c95d4c73ca None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.309754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d548ee-1148-4823-8f73-e9c95d4c73ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.314708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1320/2631] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:30.315648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de668060-9d14-4f5c-a8c3-0837e842db0e None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.315842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de668060-9d14-4f5c-a8c3-0837e842db0e None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.316064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de668060-9d14-4f5c-a8c3-0837e842db0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.338820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f7d548ee-1148-4823-8f73-e9c95d4c73ca None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:30.340423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f7d548ee-1148-4823-8f73-e9c95d4c73ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.341267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-de668060-9d14-4f5c-a8c3-0837e842db0e None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:30.342449 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-de668060-9d14-4f5c-a8c3-0837e842db0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.352912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1312/2632] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:30.353092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1321/2633] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:30.355695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d548ee-1148-4823-8f73-e9c95d4c73ca None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.356672 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de668060-9d14-4f5c-a8c3-0837e842db0e None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.367079 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d548ee-1148-4823-8f73-e9c95d4c73ca None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.369494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de668060-9d14-4f5c-a8c3-0837e842db0e None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.372387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.373189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.373468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.373737 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.374391 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:30.374692 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.376039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.376420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.376693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.377406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.377716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.378520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:30.378838 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.381056 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.397690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1313/2634] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:07:30.397894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1322/2635] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:07:30.402055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.403350 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.414767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746731a9-055c-49f2-9d59-ad5704282575 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.417113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0484e-7dc0-4dd9-85d6-99858e545246 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.420107 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.420751 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.421028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.421300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.421930 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.422895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.423605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.424036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.424382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.425114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.427416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.431488 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.473146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1323/2636] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:30.476877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1314/2637] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:30.477021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.478657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.536997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:30.552857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b2c1e-06a3-41ae-854a-a208ab5ca11b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.554011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1531187-71e5-4314-ba0b-1f2e926ba60d admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.558933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.560323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.560493 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.560803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.561086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.561316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.561696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.561827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.562150 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.562920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.564502 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.565350 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.598227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1315/2638] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:30.603143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1324/2639] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:30.603313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.604532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.613323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c6316-157d-422d-8433-f16c7e3abe7b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.619723 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.620642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.621102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.621527 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.622332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.624116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.626118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6867a9-dec1-4a7c-bd5e-30933bc783e2 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.635484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.636115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.636721 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.636998 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.638266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.640166 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.640852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1316/2640] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:30.644702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.652905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1325/2641] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:07:30.656870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.659712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a698f45-0bc5-4ad0-a5dc-e2e6ddabbabe admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.665457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.666096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.666415 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.666795 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.667694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.668403 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee4fee-f21b-4018-bf8c-a684a6a01c4c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.669624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.674918 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.675824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.676246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.676624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.677376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.679075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.682067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1317/2642] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:30.685949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.692324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1326/2643] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:07:30.696116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.704252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847b6c6-7d18-4b7d-86f2-0e9b0e7fc16c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.712083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685c6f2-a53d-4a9d-8e2a-7928ef4f8eba admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.712305 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2c1f-1095-4a40-8914-14de33fc5371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.712847 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2c1f-1095-4a40-8914-14de33fc5371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.713129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2c1f-1095-4a40-8914-14de33fc5371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.713407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2c1f-1095-4a40-8914-14de33fc5371 admin admin] PATH_INFO: `/v3/projects/fd40a6bbc4264a0a89e38c4a88e006fd/users/f6ca6cfaad41453392e54ad5a02ea8a6/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.717524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b3d2-8222-4e83-bb50-e04a20245064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.718262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b3d2-8222-4e83-bb50-e04a20245064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.718623 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b3d2-8222-4e83-bb50-e04a20245064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.719471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b3d2-8222-4e83-bb50-e04a20245064 admin admin] PATH_INFO: `/v3/projects/9bfddd295e494ec5a07023160b97b753/users/f8772eca4f824553bb980fffcf59c4c8/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.729251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2c1f-1095-4a40-8914-14de33fc5371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd40a6bbc4264a0a89e38c4a88e006fd, user_id=f6ca6cfaad41453392e54ad5a02ea8a6, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.732014 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2c1f-1095-4a40-8914-14de33fc5371 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.737890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b3d2-8222-4e83-bb50-e04a20245064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bfddd295e494ec5a07023160b97b753, user_id=f8772eca4f824553bb980fffcf59c4c8, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.740685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b3d2-8222-4e83-bb50-e04a20245064 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.750916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1318/2644] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:30 2022] PUT /identity/v3/projects/fd40a6bbc4264a0a89e38c4a88e006fd/users/f6ca6cfaad41453392e54ad5a02ea8a6/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:30.770852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.770852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.770852 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.771190 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1327/2645] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:30 2022] PUT /identity/v3/projects/9bfddd295e494ec5a07023160b97b753/users/f8772eca4f824553bb980fffcf59c4c8/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:30.772861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de729ed-214d-49d4-90c8-a0e239c25536 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.774229 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de729ed-214d-49d4-90c8-a0e239c25536 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.774551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de729ed-214d-49d4-90c8-a0e239c25536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.797388 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] MFA Rules not processed for user `f6ca6cfaad41453392e54ad5a02ea8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:30.798531 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.804780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8de729ed-214d-49d4-90c8-a0e239c25536 None None] MFA Rules not processed for user `f8772eca4f824553bb980fffcf59c4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:30.806905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8de729ed-214d-49d4-90c8-a0e239c25536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.855170 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:30.877405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1328/2646] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:30.881416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1319/2647] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:30.882621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de729ed-214d-49d4-90c8-a0e239c25536 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.894752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:30.896263 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:30.956941 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de729ed-214d-49d4-90c8-a0e239c25536 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.967302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.968079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.968578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.969078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.969504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4611aef-2b62-45ee-98d7-1ad0d606169a None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:30.969971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.972007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:30.973240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:30.973856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:30.974130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:30.974402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:30.989497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:30.991901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:31.005521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1329/2648] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:30 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:31.011486 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:31.016588 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1320/2649] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:31.025168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:31.095823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff8d93-a018-401a-bb92-1f1b3408b484 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:31.110436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:31.111271 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:31.111690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:31.111976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:31.112680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:31.115676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:31.143366 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1330/2650] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:31 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:31.147407 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:31.161464 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0cb9-ff71-4171-b178-b90c14b87a0c service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:31.168228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:31.169854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:31.170138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:31.170816 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:31.171472 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:31.173426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:31.188244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1321/2651] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:31.192589 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:31.206224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ce7c-41fb-4ee2-9e03-8146f820a840 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:31.213044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:31.213766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:31.214049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:31.214386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] PATH_INFO: `/v3/projects/ff630202b381468a86958fbed53dc829/users/ab3b188c8a854fa3b0929d76c65d5458/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:31.230340 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff630202b381468a86958fbed53dc829, user_id=ab3b188c8a854fa3b0929d76c65d5458, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:31.235097 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:31.260125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1331/2652] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:31 2022] PUT /identity/v3/projects/ff630202b381468a86958fbed53dc829/users/ab3b188c8a854fa3b0929d76c65d5458/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:31.264506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:31.282404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae2088-60fe-4405-84d0-c00cf8967f52 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:31.390731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc5cc1-e1d4-40fd-89a0-ff5f1d053239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:31.391356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5cc1-e1d4-40fd-89a0-ff5f1d053239 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:31.391635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5cc1-e1d4-40fd-89a0-ff5f1d053239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:31.391911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5cc1-e1d4-40fd-89a0-ff5f1d053239 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:31.392743 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc5cc1-e1d4-40fd-89a0-ff5f1d053239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:31.394968 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc5cc1-e1d4-40fd-89a0-ff5f1d053239 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:31.412230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1322/2653] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:07:31.412462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:31.426510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b120b-2c64-4a39-9b1c-d02e39cf933e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:31.434785 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:31.435953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:31.436430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:31.436866 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] PATH_INFO: `/v3/projects/ff630202b381468a86958fbed53dc829/users/ab3b188c8a854fa3b0929d76c65d5458/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:31.438988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff630202b381468a86958fbed53dc829, user_id=ab3b188c8a854fa3b0929d76c65d5458, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:31.443093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:31.455158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1332/2654] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:31 2022] PUT /identity/v3/projects/ff630202b381468a86958fbed53dc829/users/ab3b188c8a854fa3b0929d76c65d5458/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:31.465615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:31.465845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:31.474469 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:31.545969 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] MFA Rules not processed for user `ab3b188c8a854fa3b0929d76c65d5458`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:31.547224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:31.653894 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=100 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:31.657637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1323/2655] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:31.672481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:31.759976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e7b72-2556-4bcd-983d-79d7140051a4 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:31.765861 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:31.766501 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:31.766767 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:31.767040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:31.862462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:31.864123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:31.876460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1333/2656] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:31.919762 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:31.921377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:32.002772 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb04db3-9bcb-4f18-a3a5-b76ad458f4f9 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:32.007485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:32.008115 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:32.008426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:32.008697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] PATH_INFO: `/v3/users/5d0f55fa4717472db61860fd31f5e427` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:32.010081 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0f55fa4717472db61860fd31f5e427)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:32.011893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:32.064779 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0f55fa4717472db61860fd31f5e427'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.074040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0f55fa4717472db61860fd31f5e427'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.081294 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0f55fa4717472db61860fd31f5e427'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.081908 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0f55fa4717472db61860fd31f5e427'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.095538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0f55fa4717472db61860fd31f5e427'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.102427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1324/2657] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:31 2022] DELETE /identity/v3/users/5d0f55fa4717472db61860fd31f5e427 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:32.303553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:32.319128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e111f17-142a-405f-9b79-9ac2c0aceb1f service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:32.330821 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:32.377162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:32.377788 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:32.378064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:32.378336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] PATH_INFO: `/v3/projects/4c868216664144beb0a3a0360c56e637` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:32.379439 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c868216664144beb0a3a0360c56e637)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:32.381352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:32.426978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c868216664144beb0a3a0360c56e637', 'request_id': 'req-3e54fb75-7773-4196-b867-91c9dc16762c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.432452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c868216664144beb0a3a0360c56e637', 'request_id': 'req-3e54fb75-7773-4196-b867-91c9dc16762c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.439046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c868216664144beb0a3a0360c56e637', 'request_id': 'req-3e54fb75-7773-4196-b867-91c9dc16762c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.452974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] The token cache is being invalidate because project 4c868216664144beb0a3a0360c56e637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:32.453295 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.453923 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.455634 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1334/2658] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:32 2022] DELETE /identity/v3/projects/4c868216664144beb0a3a0360c56e637 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:32.459436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:32.461094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:32.475111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4653831-8b10-479c-8e43-2ac8781bf1f1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:32.534193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:32.534864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:32.535148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:32.535425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] PATH_INFO: `/v3/users/1000044760e545b2b5e35ef49a339953` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:32.536607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1000044760e545b2b5e35ef49a339953)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:32.538604 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:32.561815 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:32.596227 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1000044760e545b2b5e35ef49a339953'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.603512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1000044760e545b2b5e35ef49a339953'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.611557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1000044760e545b2b5e35ef49a339953'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.612204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1000044760e545b2b5e35ef49a339953'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.624814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1000044760e545b2b5e35ef49a339953'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.634535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1325/2659] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:32 2022] DELETE /identity/v3/users/1000044760e545b2b5e35ef49a339953 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:32.809093 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:32.810703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:32.829073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54fb75-7773-4196-b867-91c9dc16762c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:32.884949 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:32.885586 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:32.885863 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:32.886136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] PATH_INFO: `/v3/projects/acb03e7a0a8e4db4b42cd3ddccdd0ad8` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:32.887254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb03e7a0a8e4db4b42cd3ddccdd0ad8)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:32.889201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:32.934334 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:32.935566 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb03e7a0a8e4db4b42cd3ddccdd0ad8', 'request_id': 'req-28c1edd1-efa8-413d-a61e-a3338b3de21b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.942011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb03e7a0a8e4db4b42cd3ddccdd0ad8', 'request_id': 'req-28c1edd1-efa8-413d-a61e-a3338b3de21b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.948660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb03e7a0a8e4db4b42cd3ddccdd0ad8', 'request_id': 'req-28c1edd1-efa8-413d-a61e-a3338b3de21b'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.961018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] The token cache is being invalidate because project acb03e7a0a8e4db4b42cd3ddccdd0ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:32.961293 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.961911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-28c1edd1-efa8-413d-a61e-a3338b3de21b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:32.963873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1335/2660] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:32 2022] DELETE /identity/v3/projects/acb03e7a0a8e4db4b42cd3ddccdd0ad8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:33.710015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:33.711831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:33.725238 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debb3d1-e48a-4542-92cb-98a5f22b7fc9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:33.782849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:33.783485 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:33.783768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:33.784048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] PATH_INFO: `/v3/users/f6ca6cfaad41453392e54ad5a02ea8a6` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:33.785196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ca6cfaad41453392e54ad5a02ea8a6)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:33.787116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:33.832177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ca6cfaad41453392e54ad5a02ea8a6'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:33.839642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ca6cfaad41453392e54ad5a02ea8a6'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:33.846397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ca6cfaad41453392e54ad5a02ea8a6'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:33.847214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ca6cfaad41453392e54ad5a02ea8a6'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:33.858885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ca6cfaad41453392e54ad5a02ea8a6'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:33.864753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1326/2661] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:33 2022] DELETE /identity/v3/users/f6ca6cfaad41453392e54ad5a02ea8a6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:33.869722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:33.870205 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:33.870303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:33.890144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:33.891358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:33.951401 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1336/2662] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:33.961338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:33.963232 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:33.985849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:34.025937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061dad-26c3-4810-8a6d-6e41762a56f7 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.029623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907ea7a-031b-46be-8ce5-5095bf04bdf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.030229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907ea7a-031b-46be-8ce5-5095bf04bdf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.030512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907ea7a-031b-46be-8ce5-5095bf04bdf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.030785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907ea7a-031b-46be-8ce5-5095bf04bdf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.039695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907ea7a-031b-46be-8ce5-5095bf04bdf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.041184 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907ea7a-031b-46be-8ce5-5095bf04bdf4 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.050532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1327/2663] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:34.318115 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.319664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.334552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c452c-9a64-4297-99a9-be679d415fb7 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.340825 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.342359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.342909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.343267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] PATH_INFO: `/v3/projects/fd40a6bbc4264a0a89e38c4a88e006fd` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.344606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd40a6bbc4264a0a89e38c4a88e006fd)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.346431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.392688 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd40a6bbc4264a0a89e38c4a88e006fd', 'request_id': 'req-56699113-eb01-476e-afa7-1000adbfc045'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.398423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd40a6bbc4264a0a89e38c4a88e006fd', 'request_id': 'req-56699113-eb01-476e-afa7-1000adbfc045'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.404424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd40a6bbc4264a0a89e38c4a88e006fd', 'request_id': 'req-56699113-eb01-476e-afa7-1000adbfc045'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.419362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] The token cache is being invalidate because project fd40a6bbc4264a0a89e38c4a88e006fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:34.419658 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.420326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.421817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1337/2664] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:34 2022] DELETE /identity/v3/projects/fd40a6bbc4264a0a89e38c4a88e006fd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:34.427480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd70a6-41bc-4f28-89d4-6f136bfdf3c2 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.427764 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd70a6-41bc-4f28-89d4-6f136bfdf3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.428038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd70a6-41bc-4f28-89d4-6f136bfdf3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.448637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-93bd70a6-41bc-4f28-89d4-6f136bfdf3c2 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:34.449822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-93bd70a6-41bc-4f28-89d4-6f136bfdf3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.467476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.468985 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.512005 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1328/2665] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:34.521410 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2decc-bcaa-45fc-994d-b1e69f0afedc None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.521717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2decc-bcaa-45fc-994d-b1e69f0afedc None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.521992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2decc-bcaa-45fc-994d-b1e69f0afedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.534314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699113-eb01-476e-afa7-1000adbfc045 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.539380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.540013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.540355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.540646 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] PATH_INFO: `/v3/users/f8772eca4f824553bb980fffcf59c4c8` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.540863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-07f2decc-bcaa-45fc-994d-b1e69f0afedc None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:34.541887 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-07f2decc-bcaa-45fc-994d-b1e69f0afedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.542086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8772eca4f824553bb980fffcf59c4c8)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.543840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.545511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:34.550427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1329/2666] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:34.555180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2decc-bcaa-45fc-994d-b1e69f0afedc None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.565443 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2decc-bcaa-45fc-994d-b1e69f0afedc None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.570209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.570813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.571094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.571393 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.572628 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:34.572944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.575140 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.589302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1330/2667] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:07:34.593590 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.601192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8772eca4f824553bb980fffcf59c4c8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.602312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8772eca4f824553bb980fffcf59c4c8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.603871 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c800b7-537a-4021-9c5c-e906c783ba30 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.608621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.609252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.609593 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.609854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.610245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8772eca4f824553bb980fffcf59c4c8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.610524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.610883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8772eca4f824553bb980fffcf59c4c8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.612446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.624072 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-974146f4-532a-498f-b27c-46428a320715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8772eca4f824553bb980fffcf59c4c8'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:34.637693 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1338/2668] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:34 2022] DELETE /identity/v3/users/f8772eca4f824553bb980fffcf59c4c8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:34.641909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1331/2669] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:34 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:34.642753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405e29-2e86-472d-ae1c-026768176323 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.643037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405e29-2e86-472d-ae1c-026768176323 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.643323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405e29-2e86-472d-ae1c-026768176323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.645740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.647373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.662919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-db405e29-2e86-472d-ae1c-026768176323 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:34.664035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-db405e29-2e86-472d-ae1c-026768176323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.720690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171da15-8f30-4fd1-8c7c-6cf950e7d0ab admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.725883 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.726696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.726963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.727302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.728612 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.730843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.737389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1339/2670] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:34.748734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db405e29-2e86-472d-ae1c-026768176323 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.750297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-db405e29-2e86-472d-ae1c-026768176323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.760300 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1332/2671] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:34 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:34.761856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.773586 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c9f73-d8d4-4c68-851c-24f87ed5ac07 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.778903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.779508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.779788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.780392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.780746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.782580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.784958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:34.796356 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1333/2672] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:07:34.800124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.812525 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0b5dc-4173-4106-857f-4aa6fc8eda75 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.813972 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db405e29-2e86-472d-ae1c-026768176323 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.817802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9492a54-57e4-4ec5-9c8c-f1976f7625a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.818054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.818413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492a54-57e4-4ec5-9c8c-f1976f7625a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.818690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492a54-57e4-4ec5-9c8c-f1976f7625a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.818921 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.819031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492a54-57e4-4ec5-9c8c-f1976f7625a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.819382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.819748 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.820676 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.823127 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.829570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9492a54-57e4-4ec5-9c8c-f1976f7625a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.831019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9492a54-57e4-4ec5-9c8c-f1976f7625a9 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.834340 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1334/2673] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:07:34.838758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:34.841933 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1340/2674] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:34.849715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a96ec0-b70d-4755-8c83-f67a7ec734dc admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:34.855374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:34.856208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.856557 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.856901 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] PATH_INFO: `/v3/projects/d7d69fb708ed447e8a190efb4be9d5f1/users/226a40e02432469183e0f8b3360126dc/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.875293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d69fb708ed447e8a190efb4be9d5f1, user_id=226a40e02432469183e0f8b3360126dc, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:34.878422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:34.895838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1335/2675] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:34 2022] PUT /identity/v3/projects/d7d69fb708ed447e8a190efb4be9d5f1/users/226a40e02432469183e0f8b3360126dc/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:34.901354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:34.901857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:34.905044 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:34.926065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:34.927277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:34.954950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:35.027217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1341/2676] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:35.037627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:35.039960 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:35.116124 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12055847-784b-4c2b-9f21-a13b1ac076e4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:35.120920 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:35.121616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:35.121967 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:35.122318 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:35.133392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:35.135144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:35.146434 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1336/2677] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:35.157877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:35.159313 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:35.171849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feba58-e29b-4ffd-92ed-7bcd02d2b653 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:35.177639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:35.178651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:35.178651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:35.178870 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] PATH_INFO: `/v3/projects/9bfddd295e494ec5a07023160b97b753` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:35.179958 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bfddd295e494ec5a07023160b97b753)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:35.182535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:35.234713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bfddd295e494ec5a07023160b97b753', 'request_id': 'req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.247578 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bfddd295e494ec5a07023160b97b753', 'request_id': 'req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.261457 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bfddd295e494ec5a07023160b97b753', 'request_id': 'req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.276590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] The token cache is being invalidate because project 9bfddd295e494ec5a07023160b97b753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:35.276895 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.277547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.279051 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1342/2678] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:35 2022] DELETE /identity/v3/projects/9bfddd295e494ec5a07023160b97b753 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:35.283999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:35.295732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fed457-3d82-445e-989e-3eed9e7db942 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:35.364941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:35.365623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:35.365857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:35.366129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] PATH_INFO: `/v3/users/ab3b188c8a854fa3b0929d76c65d5458` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:35.367655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3b188c8a854fa3b0929d76c65d5458)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:35.369670 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:35.420191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3b188c8a854fa3b0929d76c65d5458'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.427076 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3b188c8a854fa3b0929d76c65d5458'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.435912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3b188c8a854fa3b0929d76c65d5458'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.436559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3b188c8a854fa3b0929d76c65d5458'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.442112 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:35.454627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-db84e0c2-2e35-4bef-9ebb-f431208a60b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3b188c8a854fa3b0929d76c65d5458'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:35.461090 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1337/2679] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:35 2022] DELETE /identity/v3/users/ab3b188c8a854fa3b0929d76c65d5458 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:35.905355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:35.906888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:35.921494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544ecd0-a282-4ce2-80d4-e9bafbaa0279 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:35.999031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:35.999760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.000112 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.000482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] PATH_INFO: `/v3/projects/ff630202b381468a86958fbed53dc829` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.001708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff630202b381468a86958fbed53dc829)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.003974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.060005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:36.063102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff630202b381468a86958fbed53dc829', 'request_id': 'req-b7604f80-074b-440c-a228-361d047591d1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:36.074319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff630202b381468a86958fbed53dc829', 'request_id': 'req-b7604f80-074b-440c-a228-361d047591d1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:36.080843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff630202b381468a86958fbed53dc829', 'request_id': 'req-b7604f80-074b-440c-a228-361d047591d1'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:36.096645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] The token cache is being invalidate because project ff630202b381468a86958fbed53dc829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:36.097020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:36.097723 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b7604f80-074b-440c-a228-361d047591d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:36.100211 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1343/2680] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:35 2022] DELETE /identity/v3/projects/ff630202b381468a86958fbed53dc829 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:36.106464 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06304c7-2a64-4262-a456-edcc5ef6e37f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.107464 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06304c7-2a64-4262-a456-edcc5ef6e37f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.107759 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06304c7-2a64-4262-a456-edcc5ef6e37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.127896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-f06304c7-2a64-4262-a456-edcc5ef6e37f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:36.129160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f06304c7-2a64-4262-a456-edcc5ef6e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:36.198767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1338/2681] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:36.207007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51b62e-121a-4254-83c3-dcfb0c44c312 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.207303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51b62e-121a-4254-83c3-dcfb0c44c312 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.207580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51b62e-121a-4254-83c3-dcfb0c44c312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.230823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-5a51b62e-121a-4254-83c3-dcfb0c44c312 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:36.232046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-5a51b62e-121a-4254-83c3-dcfb0c44c312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:36.240645 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1344/2682] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:36.244885 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06304c7-2a64-4262-a456-edcc5ef6e37f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:36.255284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06304c7-2a64-4262-a456-edcc5ef6e37f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:36.260800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:36.261404 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.261708 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.261984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.262627 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:36.262931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.264811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.281616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1339/2683] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:07:36.281771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51b62e-121a-4254-83c3-dcfb0c44c312 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:36.296228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51b62e-121a-4254-83c3-dcfb0c44c312 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:36.303676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:36.304345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.304902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.305041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.305739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.308250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.349216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:36.350232 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1345/2684] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:36 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:36.361909 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f5cd5-9668-4f6e-9cf8-d50b0b5df10e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:36.448564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:36.449228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.449518 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.449797 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.450644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.453745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.483274 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1340/2685] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:36 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:36.487448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:36.512427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91574801-1790-496f-b36f-df4d065d08cd admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:36.519154 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:36.520701 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.520988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.521267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.522545 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.524210 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.540648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1346/2686] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:36.542188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:36.552943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eedfb6-f604-444f-800d-fbd736a9cb69 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:36.559991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5942475-5175-4afd-a15a-b1d109ab0a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:36.560997 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5942475-5175-4afd-a15a-b1d109ab0a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.561365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5942475-5175-4afd-a15a-b1d109ab0a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.562136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5942475-5175-4afd-a15a-b1d109ab0a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.562446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5942475-5175-4afd-a15a-b1d109ab0a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.564182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5942475-5175-4afd-a15a-b1d109ab0a7a admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.587547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1341/2687] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:36.589590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:36.601374 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23a54-72f0-4d64-98d0-de40ad7bc10b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:36.607444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:36.608305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.608692 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.609006 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] PATH_INFO: `/v3/projects/9490f3092ec94727a318a46d84c0fc98/users/6a9d217086294197b4e0e870ba767e33/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.626277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9490f3092ec94727a318a46d84c0fc98, user_id=6a9d217086294197b4e0e870ba767e33, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.629114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.645729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1347/2688] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:36 2022] PUT /identity/v3/projects/9490f3092ec94727a318a46d84c0fc98/users/6a9d217086294197b4e0e870ba767e33/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:36.657474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1a810-d7bb-4643-ba86-22f6f0812561 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.657905 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1a810-d7bb-4643-ba86-22f6f0812561 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.658383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1a810-d7bb-4643-ba86-22f6f0812561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.722851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-27b1a810-d7bb-4643-ba86-22f6f0812561 None None] MFA Rules not processed for user `6a9d217086294197b4e0e870ba767e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:36.724832 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-27b1a810-d7bb-4643-ba86-22f6f0812561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:36.784792 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:36.825289 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1342/2689] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:36.834045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:36.836364 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:36.918398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f6a7-4a2c-41e9-b0a7-4e88772ab345 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:36.920950 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6d9d4-aa3d-498e-870f-e236d2d8fce7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:36.921642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d9d4-aa3d-498e-870f-e236d2d8fce7 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:36.922007 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d9d4-aa3d-498e-870f-e236d2d8fce7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:36.922969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d9d4-aa3d-498e-870f-e236d2d8fce7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:36.955803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6d9d4-aa3d-498e-870f-e236d2d8fce7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:36.957484 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6d9d4-aa3d-498e-870f-e236d2d8fce7 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:36.968079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1348/2690] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:37.000856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1a810-d7bb-4643-ba86-22f6f0812561 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:37.077390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1a810-d7bb-4643-ba86-22f6f0812561 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:37.082195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:37.082842 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:37.083126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:37.083402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] PATH_INFO: `/v3/users/6a9d217086294197b4e0e870ba767e33` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:37.084647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9d217086294197b4e0e870ba767e33)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:37.088025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:37.141519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9d217086294197b4e0e870ba767e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.148582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9d217086294197b4e0e870ba767e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.155980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9d217086294197b4e0e870ba767e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.156651 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9d217086294197b4e0e870ba767e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.170486 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9d217086294197b4e0e870ba767e33'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.176602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1343/2691] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:36 2022] DELETE /identity/v3/users/6a9d217086294197b4e0e870ba767e33 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:37.181744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:37.182118 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:37.182398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:37.202587 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:37.203802 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:37.267954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1349/2692] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:37.278174 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:37.279738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:37.347903 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8ba6e-576e-4d70-9fec-29e27cf190a3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:37.353923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04bcae-b109-49a9-bf38-c99497ef2c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:37.356283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04bcae-b109-49a9-bf38-c99497ef2c23 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:37.356643 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04bcae-b109-49a9-bf38-c99497ef2c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:37.357009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04bcae-b109-49a9-bf38-c99497ef2c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:37.368041 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04bcae-b109-49a9-bf38-c99497ef2c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:37.369838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04bcae-b109-49a9-bf38-c99497ef2c23 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:37.381293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1344/2693] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:37.777906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:37.782328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:37.801452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4605e00-5458-4a82-842d-8bb56f56486d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:37.808018 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:37.808871 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:37.809223 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:37.809570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] PATH_INFO: `/v3/projects/9490f3092ec94727a318a46d84c0fc98` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:37.814653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9490f3092ec94727a318a46d84c0fc98)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:37.816739 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:37.831547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:37.883215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9490f3092ec94727a318a46d84c0fc98', 'request_id': 'req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.890912 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9490f3092ec94727a318a46d84c0fc98', 'request_id': 'req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.899775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9490f3092ec94727a318a46d84c0fc98', 'request_id': 'req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.918086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] The token cache is being invalidate because project 9490f3092ec94727a318a46d84c0fc98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:37.918414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.919060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:37.920915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1350/2694] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:37 2022] DELETE /identity/v3/projects/9490f3092ec94727a318a46d84c0fc98 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:40.354551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa888-d1aa-4eca-899d-3cfca5571f33 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:40.355086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa888-d1aa-4eca-899d-3cfca5571f33 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:40.355203 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa888-d1aa-4eca-899d-3cfca5571f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:40.362514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:40.389446 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3a6fa888-d1aa-4eca-899d-3cfca5571f33 None None] MFA Rules not processed for user `226a40e02432469183e0f8b3360126dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:40.390697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3a6fa888-d1aa-4eca-899d-3cfca5571f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:40.499925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1345/2695] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:40.504423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:40.505911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:40.582710 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3302f1d-cd7d-4bb7-99aa-3e1188ba6e63 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:40.587778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:40.588511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:40.589144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:40.589491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:40.590199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:40.592187 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:40.626468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1351/2696] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:40 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:40.628039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fa888-d1aa-4eca-899d-3cfca5571f33 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:40.640667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fa888-d1aa-4eca-899d-3cfca5571f33 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:40.703282 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:40.703828 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:40.704111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:40.704421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:40.705071 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:40.707089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:40.736951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1346/2697] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:40 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:40.740733 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:40.754395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f32cd-0899-4e52-8e0e-6ca2b768de9f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:40.766055 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:40.767243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:40.767671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:40.767976 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:40.768735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:40.770434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:40.786539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1352/2698] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:40.791409 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:40.802747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94125749-fa40-488a-8eb7-28c86001c497 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:40.808953 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:40.809778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:40.810091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:40.810414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] PATH_INFO: `/v3/projects/53ed071861ac4e5e8bbaa0f9152c711e/users/b24535c8baf24b3d8ca6ceff8780238f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:40.829635 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ed071861ac4e5e8bbaa0f9152c711e, user_id=b24535c8baf24b3d8ca6ceff8780238f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:40.833045 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:40.850032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1347/2699] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:40 2022] PUT /identity/v3/projects/53ed071861ac4e5e8bbaa0f9152c711e/users/b24535c8baf24b3d8ca6ceff8780238f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:40.854673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:40.866299 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2a7a6-51fe-4b8a-8e4c-a9c79885f81f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:40.933636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:40.934268 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:40.934549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:40.934823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:40.935480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:40.937121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:40.948619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1353/2700] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:40.952979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:40.964468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338d9ea-372f-4f8b-8d07-67b3255f68e1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:40.971614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd882c-d2ba-4922-86fb-4e4ddd846924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:40.972339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd882c-d2ba-4922-86fb-4e4ddd846924 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:40.972624 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd882c-d2ba-4922-86fb-4e4ddd846924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:40.972900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd882c-d2ba-4922-86fb-4e4ddd846924 admin admin] PATH_INFO: `/v3/projects/53ed071861ac4e5e8bbaa0f9152c711e/users/b24535c8baf24b3d8ca6ceff8780238f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:40.975173 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd882c-d2ba-4922-86fb-4e4ddd846924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ed071861ac4e5e8bbaa0f9152c711e, user_id=b24535c8baf24b3d8ca6ceff8780238f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:40.978177 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd882c-d2ba-4922-86fb-4e4ddd846924 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:40.990800 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1348/2701] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:40 2022] PUT /identity/v3/projects/53ed071861ac4e5e8bbaa0f9152c711e/users/b24535c8baf24b3d8ca6ceff8780238f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:43.871256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:43.873431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:43.888373 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:43.968510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7aa8-810d-4773-bd94-5827578795f3 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:44.072101 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:44.072965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:44.073585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:44.073724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:44.075740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:44.184864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:44.186342 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:44.205833 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1354/2702] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:45.671716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80077358-1c61-4061-9bd7-a3b0aaecbe2a None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:45.672162 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80077358-1c61-4061-9bd7-a3b0aaecbe2a None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:45.672276 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80077358-1c61-4061-9bd7-a3b0aaecbe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:45.708101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-80077358-1c61-4061-9bd7-a3b0aaecbe2a None None] MFA Rules not processed for user `b24535c8baf24b3d8ca6ceff8780238f`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:45.709702 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-80077358-1c61-4061-9bd7-a3b0aaecbe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:45.750273 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:45.802631 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1349/2703] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:45.813707 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:45.815883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:45.959049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e660de-968e-4c29-bd4d-1729c770ec5e service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:45.972835 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c76a4a9747469d88a0a1a5748452a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:45.974520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:45.974984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:45.975429 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:45.986337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:46.103919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:46.107261 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:46.120509 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1355/2704] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:46.302156 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:46.302753 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:46.305284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:46.342583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:46.346408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:46.424306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1350/2705] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 17:07:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:46.485799 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:46.487807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:46.572404 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6ee1-4e99-47c6-9f46-133d40c7ee1b service glance] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:46.579107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:46.580177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:46.580490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:46.580859 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:46.593021 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:46.595987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:46.606752 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1356/2706] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:07:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:07:48.834421 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:48.843470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:48.866767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a87146-4a27-43f1-a64c-28440ca22f8e None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:48.878636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:48.879128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:48.879653 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:48.880381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] PATH_INFO: `/v3/users/96495ad2ad764ba3a0fb1e3bc28fd8bd` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:48.882694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96495ad2ad764ba3a0fb1e3bc28fd8bd)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:48.885200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:48.970479 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96495ad2ad764ba3a0fb1e3bc28fd8bd'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:48.980389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96495ad2ad764ba3a0fb1e3bc28fd8bd'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:48.992882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96495ad2ad764ba3a0fb1e3bc28fd8bd'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:48.993727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96495ad2ad764ba3a0fb1e3bc28fd8bd'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:49.010849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ca5b1599-a92c-44b8-88cd-372ecdaba510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96495ad2ad764ba3a0fb1e3bc28fd8bd'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:49.018454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1351/2707] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:48 2022] DELETE /identity/v3/users/96495ad2ad764ba3a0fb1e3bc28fd8bd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:49.280475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:49.304471 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74485a-5e6d-4823-9997-ec81bc528002 service swift] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:49.388243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:49.388939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.389397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.389880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] PATH_INFO: `/v3/projects/d17c2f74324146ca932ff151c9f7a003` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.391248 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17c2f74324146ca932ff151c9f7a003)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:49.393583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:49.462654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:49.479382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17c2f74324146ca932ff151c9f7a003', 'request_id': 'req-674a1498-48c3-4da9-913e-0209d7535784'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:49.490069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17c2f74324146ca932ff151c9f7a003', 'request_id': 'req-674a1498-48c3-4da9-913e-0209d7535784'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:49.497134 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17c2f74324146ca932ff151c9f7a003', 'request_id': 'req-674a1498-48c3-4da9-913e-0209d7535784'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:49.515810 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] The token cache is being invalidate because project d17c2f74324146ca932ff151c9f7a003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:49.516099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:49.516812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-674a1498-48c3-4da9-913e-0209d7535784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:49.518528 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1357/2708] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:49 2022] DELETE /identity/v3/projects/d17c2f74324146ca932ff151c9f7a003 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:49.523204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e010fb-6950-400a-9257-7c01540612d5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.523652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e010fb-6950-400a-9257-7c01540612d5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.523927 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e010fb-6950-400a-9257-7c01540612d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.552216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-67e010fb-6950-400a-9257-7c01540612d5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:49.553438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-67e010fb-6950-400a-9257-7c01540612d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:49.627220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:49.663597 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1352/2709] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:49.665460 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee87a-c5a5-40b3-806a-46095575567a None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.665796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee87a-c5a5-40b3-806a-46095575567a None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.666109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee87a-c5a5-40b3-806a-46095575567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.689973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3c4ee87a-c5a5-40b3-806a-46095575567a None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:49.691725 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3c4ee87a-c5a5-40b3-806a-46095575567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:49.702119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1358/2710] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:49.706209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e010fb-6950-400a-9257-7c01540612d5 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:49.718101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e010fb-6950-400a-9257-7c01540612d5 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:49.723379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:49.724000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.724321 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.724599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.725833 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:49.726144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:49.728617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:49.748200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1353/2711] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:07:49.750668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ee87a-c5a5-40b3-806a-46095575567a None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:49.764121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ee87a-c5a5-40b3-806a-46095575567a None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:49.770256 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:49.770876 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.771154 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.771424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.772104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:49.774123 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:49.822209 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1359/2712] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:49 2022] POST /identity/v3/projects => generated 403 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:49.822857 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:49.840283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8b889-4cf5-4f1b-b268-dd84c7e59200 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:49.921466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:49.922846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.923191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.923618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.925520 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:49.926420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:49.965311 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1354/2713] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:49 2022] POST /identity/v3/users => generated 302 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:49.966613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:49.978258 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626bba4-0a18-46fc-b8b3-6839f3ba0a07 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:49.986286 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:49.987066 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.987247 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.987888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.988779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:49.990388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:50.002153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1360/2714] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:50.006257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:50.019380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b37df-a2a9-41c5-aa84-009ec5034a3c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:50.025664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f84ca-431c-4f01-a27f-dab9a56b7791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:50.026284 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f84ca-431c-4f01-a27f-dab9a56b7791 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:50.026568 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f84ca-431c-4f01-a27f-dab9a56b7791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:50.026841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f84ca-431c-4f01-a27f-dab9a56b7791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:50.027492 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f84ca-431c-4f01-a27f-dab9a56b7791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:50.029130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f84ca-431c-4f01-a27f-dab9a56b7791 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:50.043811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1355/2715] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:50.048598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:50.060864 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c423-9ba5-458c-9266-4d44ff39cd0f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:50.065829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:50.066812 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:50.067175 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:50.067544 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] PATH_INFO: `/v3/projects/219a72a019db4fc8ac830a5ad806b647/users/53e9e696fb724193b4beccbb85a2098d/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:50.086068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=219a72a019db4fc8ac830a5ad806b647, user_id=53e9e696fb724193b4beccbb85a2098d, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:50.089133 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:50.110206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1361/2716] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:50 2022] PUT /identity/v3/projects/219a72a019db4fc8ac830a5ad806b647/users/53e9e696fb724193b4beccbb85a2098d/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:50.115287 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0edfa8-d10c-4a17-9d3f-618089ff3c61 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:50.115850 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0edfa8-d10c-4a17-9d3f-618089ff3c61 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:50.116331 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0edfa8-d10c-4a17-9d3f-618089ff3c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:50.137985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2e0edfa8-d10c-4a17-9d3f-618089ff3c61 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:50.140055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2e0edfa8-d10c-4a17-9d3f-618089ff3c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:50.227912 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1356/2717] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:50.239605 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:50.241388 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:50.343059 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2351dbd-6cf0-42e7-a177-4df414772a9b admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:50.348607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:50.353157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:50.353755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:50.354409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:50.366379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:50.369535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:50.386508 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1362/2718] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:51.270032 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3800c7-fb14-4b8e-be72-fa232a8e96b6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.270587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3800c7-fb14-4b8e-be72-fa232a8e96b6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.270690 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3800c7-fb14-4b8e-be72-fa232a8e96b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.313936 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-6e3800c7-fb14-4b8e-be72-fa232a8e96b6 None None] MFA Rules not processed for user `53e9e696fb724193b4beccbb85a2098d`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:51.315207 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-6e3800c7-fb14-4b8e-be72-fa232a8e96b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:51.395423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:51.443060 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1357/2719] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:51 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:51.444331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:51.470562 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4240fd-a4db-46c5-9a7e-4768c770f633 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:51.481843 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:51.482722 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.483111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.483624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.484824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:51.487341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:51.555147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1363/2720] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:51 2022] POST /identity/v3/projects => generated 403 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:51.559200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3800c7-fb14-4b8e-be72-fa232a8e96b6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:51.574766 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3800c7-fb14-4b8e-be72-fa232a8e96b6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:51.691235 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:51.692028 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.692458 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.692899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.693632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:51.695701 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:51.727986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1358/2721] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:51 2022] POST /identity/v3/users => generated 308 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:51.732005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:51.755629 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7c435-800e-4c35-9afb-376f4a54dfc0 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:51.764354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:51.764993 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.765526 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.765699 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.766323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:51.767966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:51.785191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1364/2722] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:51.789113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:51.802098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694dcb-6694-4ad1-ad52-2959a4ce388c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:51.808716 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:51.809580 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.809971 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.810442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] PATH_INFO: `/v3/projects/a42e7d2d45b742a3a124f8c50c6f4078/users/2d7fc1530ac04a1f9d22f4232116835f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.848718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42e7d2d45b742a3a124f8c50c6f4078, user_id=2d7fc1530ac04a1f9d22f4232116835f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:51.854851 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:51.884056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1359/2723] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:51 2022] PUT /identity/v3/projects/a42e7d2d45b742a3a124f8c50c6f4078/users/2d7fc1530ac04a1f9d22f4232116835f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:51.885428 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:51.897356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81f0b6-bde5-4cdc-9d29-270bcf8a7d4a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:51.966078 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:51.996991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96112026-70fc-418b-9708-20918cc7302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:51.997730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112026-70fc-418b-9708-20918cc7302f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.998079 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112026-70fc-418b-9708-20918cc7302f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.998432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112026-70fc-418b-9708-20918cc7302f admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.999136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96112026-70fc-418b-9708-20918cc7302f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:52.000854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96112026-70fc-418b-9708-20918cc7302f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:52.015715 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1365/2724] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:52.019868 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:52.031438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a8434-c6ad-4f23-b193-64b6ab80a8f0 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:52.043165 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:52.043862 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:52.044176 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:52.044452 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] PATH_INFO: `/v3/projects/a42e7d2d45b742a3a124f8c50c6f4078/users/2d7fc1530ac04a1f9d22f4232116835f/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:52.051000 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42e7d2d45b742a3a124f8c50c6f4078, user_id=2d7fc1530ac04a1f9d22f4232116835f, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:52.053767 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:52.078119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1360/2725] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:52 2022] PUT /identity/v3/projects/a42e7d2d45b742a3a124f8c50c6f4078/users/2d7fc1530ac04a1f9d22f4232116835f/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:52.820465 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce4f04-3d7a-4856-a9dc-b803e8dec491 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:52.822191 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce4f04-3d7a-4856-a9dc-b803e8dec491 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:52.822615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce4f04-3d7a-4856-a9dc-b803e8dec491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:52.855064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-8cce4f04-3d7a-4856-a9dc-b803e8dec491 None None] MFA Rules not processed for user `2d7fc1530ac04a1f9d22f4232116835f`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:52.856742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8cce4f04-3d7a-4856-a9dc-b803e8dec491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:52.964987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1366/2726] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:54.234400 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:54.308373 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9f0f9-625c-4914-9b9e-bb455b542100 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:54.316380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:54.317733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:54.318320 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:54.318923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] PATH_INFO: `/v3/users/53e9e696fb724193b4beccbb85a2098d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:54.321649 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e9e696fb724193b4beccbb85a2098d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:54.323727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:54.372551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:54.388768 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e9e696fb724193b4beccbb85a2098d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.397814 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e9e696fb724193b4beccbb85a2098d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.410358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e9e696fb724193b4beccbb85a2098d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.411166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e9e696fb724193b4beccbb85a2098d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.426970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e9e696fb724193b4beccbb85a2098d'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.436471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1361/2727] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:54 2022] DELETE /identity/v3/users/53e9e696fb724193b4beccbb85a2098d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:07:54.637480 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce4f04-3d7a-4856-a9dc-b803e8dec491 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:54.656348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce4f04-3d7a-4856-a9dc-b803e8dec491 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:54.745122 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:54.745772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:54.746053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:54.746344 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] PATH_INFO: `/v3/projects/219a72a019db4fc8ac830a5ad806b647` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:54.747506 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=219a72a019db4fc8ac830a5ad806b647)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:54.749582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:54.816620 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '219a72a019db4fc8ac830a5ad806b647', 'request_id': 'req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.823167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '219a72a019db4fc8ac830a5ad806b647', 'request_id': 'req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.824414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:54.837216 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '219a72a019db4fc8ac830a5ad806b647', 'request_id': 'req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.854144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] The token cache is being invalidate because project 219a72a019db4fc8ac830a5ad806b647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:54.854553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.855236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:54.856816 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1367/2728] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:54 2022] DELETE /identity/v3/projects/219a72a019db4fc8ac830a5ad806b647 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:54.860876 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:54.862454 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:54.876995 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb6dd0-13d9-4b76-881b-2b3d4b4d55f3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:54.957622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:54.958348 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:54.958699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:54.959048 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] PATH_INFO: `/v3/users/2d7fc1530ac04a1f9d22f4232116835f` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:54.960240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7fc1530ac04a1f9d22f4232116835f)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:54.962506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.017765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7fc1530ac04a1f9d22f4232116835f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.024966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7fc1530ac04a1f9d22f4232116835f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.033864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7fc1530ac04a1f9d22f4232116835f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.034528 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7fc1530ac04a1f9d22f4232116835f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.052773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-02244cf9-0f7a-4450-a3dc-d35808ab50d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7fc1530ac04a1f9d22f4232116835f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.060166 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1362/2729] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:07:54 2022] DELETE /identity/v3/users/2d7fc1530ac04a1f9d22f4232116835f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:55.283370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.285492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:55.313965 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f57c8-2ce1-4810-ba2a-0d2f43ae98d5 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.397150 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.398113 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.398590 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.399615 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] PATH_INFO: `/v3/projects/a42e7d2d45b742a3a124f8c50c6f4078` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.400953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42e7d2d45b742a3a124f8c50c6f4078)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.403086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.462354 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42e7d2d45b742a3a124f8c50c6f4078', 'request_id': 'req-b6f77e4b-df14-4869-8840-9174b051798d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.468107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42e7d2d45b742a3a124f8c50c6f4078', 'request_id': 'req-b6f77e4b-df14-4869-8840-9174b051798d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.475580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42e7d2d45b742a3a124f8c50c6f4078', 'request_id': 'req-b6f77e4b-df14-4869-8840-9174b051798d'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.493674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] The token cache is being invalidate because project a42e7d2d45b742a3a124f8c50c6f4078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:07:55.494063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.494797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-b6f77e4b-df14-4869-8840-9174b051798d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:07:55.496425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1368/2730] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:07:55 2022] DELETE /identity/v3/projects/a42e7d2d45b742a3a124f8c50c6f4078 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:55.501484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117714-91bd-4cf4-a8e1-66afe9ae8669 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.502008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117714-91bd-4cf4-a8e1-66afe9ae8669 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.502008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117714-91bd-4cf4-a8e1-66afe9ae8669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.522636 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-14117714-91bd-4cf4-a8e1-66afe9ae8669 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:55.524392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-14117714-91bd-4cf4-a8e1-66afe9ae8669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:55.556632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:55.596138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1363/2731] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:55.605179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062188-89ec-4aa9-baf4-70ba29daedf5 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.605470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062188-89ec-4aa9-baf4-70ba29daedf5 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.605748 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062188-89ec-4aa9-baf4-70ba29daedf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.630768 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-b9062188-89ec-4aa9-baf4-70ba29daedf5 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:55.632011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-b9062188-89ec-4aa9-baf4-70ba29daedf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:55.640568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1369/2732] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:55.644845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14117714-91bd-4cf4-a8e1-66afe9ae8669 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.656286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14117714-91bd-4cf4-a8e1-66afe9ae8669 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.661390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.662008 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.662312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.662569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.663217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:07:55.663538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.665724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.682343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1364/2733] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:07:55.683524 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062188-89ec-4aa9-baf4-70ba29daedf5 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.697103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062188-89ec-4aa9-baf4-70ba29daedf5 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.702842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.703801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.704075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.704387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.705062 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.707259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.752338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1370/2734] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:55 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:55.753569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.764730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaea80-64c7-40d9-99a3-b76b2f4ba711 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.839037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.839660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.839989 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.840299 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.840990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.842938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.868466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1365/2735] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:55 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:07:55.872359 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.883910 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a021e3f-be5d-4db6-bf6d-32c7217e9c98 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.889060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.889797 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.890139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.890542 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.891281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.893068 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.904431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1371/2736] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:07:55.908302 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.922114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3780ca-668d-484d-95cf-76fe349f7498 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.932215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfc415-8047-400c-9f19-79c03a4dde90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.932810 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfc415-8047-400c-9f19-79c03a4dde90 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.933096 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfc415-8047-400c-9f19-79c03a4dde90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.933379 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfc415-8047-400c-9f19-79c03a4dde90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.934034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfc415-8047-400c-9f19-79c03a4dde90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.935647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfc415-8047-400c-9f19-79c03a4dde90 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.949066 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1366/2737] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:55.952924 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.964035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331a3a5-936c-408d-920e-01acd6923e86 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.970392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.971076 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.971349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.971619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] PATH_INFO: `/v3/projects/1341c99aafdd44a98cd7c1c375035dd0/users/400ea105088744c291628d6d3be26b11/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.989743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1341c99aafdd44a98cd7c1c375035dd0, user_id=400ea105088744c291628d6d3be26b11, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.992482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.011936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1372/2738] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:55 2022] PUT /identity/v3/projects/1341c99aafdd44a98cd7c1c375035dd0/users/400ea105088744c291628d6d3be26b11/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:56.022544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba8945-3949-4b6e-a937-4832a7fbe20b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.022830 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba8945-3949-4b6e-a937-4832a7fbe20b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.023101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba8945-3949-4b6e-a937-4832a7fbe20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.062733 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-69ba8945-3949-4b6e-a937-4832a7fbe20b None None] MFA Rules not processed for user `400ea105088744c291628d6d3be26b11`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:56.064205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-69ba8945-3949-4b6e-a937-4832a7fbe20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:56.147630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1367/2739] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:56.154444 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:56.168157 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:56.229890 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d2ec6-9ed0-49f9-af50-34c8c957bc4c admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:56.236309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:56.236309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.237102 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.237239 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.237883 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:56.239840 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.276183 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1373/2740] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:07:56 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:56.280058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba8945-3949-4b6e-a937-4832a7fbe20b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:56.295569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba8945-3949-4b6e-a937-4832a7fbe20b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:56.326116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:56.369605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:56.369605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.369605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.369605 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.370163 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:56.371984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.397916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1368/2741] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:07:56 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:07:56.402107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:56.416980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acd4a2-8687-4236-aeab-d89a823c4222 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:56.421856 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:56.422546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.422901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.423251 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.428680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:56.428680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.441107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1374/2742] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:56.445376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:56.459147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd2740-82ae-4fd3-ae5a-6b551cd69eaf admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:56.463058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:56.463735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.464015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.464399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] PATH_INFO: `/v3/projects/12ddde56409a4ea784568b457047d05f/users/9c5f8637b02947eb8fa5c9447544c105/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.483376 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ddde56409a4ea784568b457047d05f, user_id=9c5f8637b02947eb8fa5c9447544c105, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:56.486198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.503864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1369/2743] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:56 2022] PUT /identity/v3/projects/12ddde56409a4ea784568b457047d05f/users/9c5f8637b02947eb8fa5c9447544c105/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:56.505105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:56.517483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e21073-04ea-49fe-bd2d-c0eb661d7202 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:56.583987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc972a-2671-4007-b146-a9f2bd1fe180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:56.584665 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc972a-2671-4007-b146-a9f2bd1fe180 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.584947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc972a-2671-4007-b146-a9f2bd1fe180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.585222 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc972a-2671-4007-b146-a9f2bd1fe180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.586136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc972a-2671-4007-b146-a9f2bd1fe180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:56.587865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc972a-2671-4007-b146-a9f2bd1fe180 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.610778 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1375/2744] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:07:56.612249 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:56.624394 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ebb68-31c5-4db5-b204-edf5b3a492da admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:56.629519 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:56.630306 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.630660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.631007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] PATH_INFO: `/v3/projects/12ddde56409a4ea784568b457047d05f/users/9c5f8637b02947eb8fa5c9447544c105/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.633432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ddde56409a4ea784568b457047d05f, user_id=9c5f8637b02947eb8fa5c9447544c105, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:56.636310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.648138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1370/2745] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:07:56 2022] PUT /identity/v3/projects/12ddde56409a4ea784568b457047d05f/users/9c5f8637b02947eb8fa5c9447544c105/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:07:56.658877 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbefbfc-44ca-46e5-893a-17fb47e5e214 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.659436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbefbfc-44ca-46e5-893a-17fb47e5e214 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.659436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbefbfc-44ca-46e5-893a-17fb47e5e214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.691463 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9cbefbfc-44ca-46e5-893a-17fb47e5e214 None None] MFA Rules not processed for user `9c5f8637b02947eb8fa5c9447544c105`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:56.692763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9cbefbfc-44ca-46e5-893a-17fb47e5e214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:56.786031 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1376/2746] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:56.797389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:56.799077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:56.863964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481403f-fdc6-4220-8fe4-8c046d5290f9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:56.868139 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418bcf5-d24d-4526-ae66-da6c2c1c4e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:56.868796 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418bcf5-d24d-4526-ae66-da6c2c1c4e51 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.869075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418bcf5-d24d-4526-ae66-da6c2c1c4e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.869353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418bcf5-d24d-4526-ae66-da6c2c1c4e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.881128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418bcf5-d24d-4526-ae66-da6c2c1c4e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:56.882591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418bcf5-d24d-4526-ae66-da6c2c1c4e51 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:56.900280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1371/2747] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:56.997360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbefbfc-44ca-46e5-893a-17fb47e5e214 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:57.007742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbefbfc-44ca-46e5-893a-17fb47e5e214 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:57.016520 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:57.017267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:57.017674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:57.018050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:57.025094 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:57.083382 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:57.085142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:57.096370 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1377/2748] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:57.412473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fcdb3-5363-4990-822d-a74df574ffa5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:57.412693 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fcdb3-5363-4990-822d-a74df574ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:57.413374 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fcdb3-5363-4990-822d-a74df574ffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:57.434608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-a94fcdb3-5363-4990-822d-a74df574ffa5 None None] MFA Rules not processed for user `226a40e02432469183e0f8b3360126dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:57.435821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a94fcdb3-5363-4990-822d-a74df574ffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:57.490266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1372/2749] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:07:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:57.738277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:57.739697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:57.794632 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c8ab2-4439-4be4-9e82-60f905419ca5 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:57.798212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:57.798808 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:57.799084 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:57.799352 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:57.801026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:57.810727 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:57.812125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:57.819893 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1378/2750] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:59.719639 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7a49e-458b-4f8c-b835-7439a152a4e0 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:59.720224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7a49e-458b-4f8c-b835-7439a152a4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:59.720224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7a49e-458b-4f8c-b835-7439a152a4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:59.741914 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ebe7a49e-458b-4f8c-b835-7439a152a4e0 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:59.743970 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ebe7a49e-458b-4f8c-b835-7439a152a4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:59.780034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:59.820033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1373/2751] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 10 17:07:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:59.833624 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:59.835110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:59.917630 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681abd8f-9599-4f0e-bf58-340670608ca7 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:59.922982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7d3c2-eb62-48b5-8b9d-59192429efcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:59.923594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7d3c2-eb62-48b5-8b9d-59192429efcf service swift] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:59.924041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7d3c2-eb62-48b5-8b9d-59192429efcf service swift] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:59.924367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7d3c2-eb62-48b5-8b9d-59192429efcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:59.938038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7d3c2-eb62-48b5-8b9d-59192429efcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:59.940168 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7d3c2-eb62-48b5-8b9d-59192429efcf service swift] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:59.951236 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1379/2752] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:07:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:08:09.812615 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=12 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:09.947687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=82 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:12.371108 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7a49e-458b-4f8c-b835-7439a152a4e0 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:12.439933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7a49e-458b-4f8c-b835-7439a152a4e0 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:12.444445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:12.445728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:12.446006 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:12.446381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] PATH_INFO: `/v3/users/400ea105088744c291628d6d3be26b11` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:12.447516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=400ea105088744c291628d6d3be26b11)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:12.449467 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:12.504493 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400ea105088744c291628d6d3be26b11'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:12.511938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400ea105088744c291628d6d3be26b11'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:12.519886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400ea105088744c291628d6d3be26b11'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:12.520564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400ea105088744c291628d6d3be26b11'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:12.533195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400ea105088744c291628d6d3be26b11'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:12.539645 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1374/2753] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:12 2022] DELETE /identity/v3/users/400ea105088744c291628d6d3be26b11 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:12.544008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:12.544492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:12.544771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:12.564577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:12.566213 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:12.629228 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1380/2754] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:08:12.638898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:12.640355 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:12.702559 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d07fb-3711-4d95-9a20-8d76ad413e12 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:12.706268 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:12.706869 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:12.707145 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:12.707442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:12.718535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:12.719997 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:12.730956 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1375/2755] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:08:13.018220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:13.019611 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:13.032188 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36122f-b4df-447f-84be-a393c15c4688 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:13.036839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:13.037477 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.037755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.038026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] PATH_INFO: `/v3/projects/1341c99aafdd44a98cd7c1c375035dd0` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.039865 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1341c99aafdd44a98cd7c1c375035dd0)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:13.043516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:13.103320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1341c99aafdd44a98cd7c1c375035dd0', 'request_id': 'req-29e2b80f-bedb-466c-b73b-77b124331a10'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.109324 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1341c99aafdd44a98cd7c1c375035dd0', 'request_id': 'req-29e2b80f-bedb-466c-b73b-77b124331a10'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.115659 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1341c99aafdd44a98cd7c1c375035dd0', 'request_id': 'req-29e2b80f-bedb-466c-b73b-77b124331a10'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.128500 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] The token cache is being invalidate because project 1341c99aafdd44a98cd7c1c375035dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:13.128789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.129402 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.130880 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1381/2756] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:13 2022] DELETE /identity/v3/projects/1341c99aafdd44a98cd7c1c375035dd0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:13.134964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:13.145209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a88ef-57c0-4833-9334-4966307f5c3a service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:13.177261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:13.219774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:13.220352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.220630 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.220899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] PATH_INFO: `/v3/users/9c5f8637b02947eb8fa5c9447544c105` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.222012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c5f8637b02947eb8fa5c9447544c105)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:13.223958 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:13.275027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c5f8637b02947eb8fa5c9447544c105'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.282087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c5f8637b02947eb8fa5c9447544c105'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.291201 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c5f8637b02947eb8fa5c9447544c105'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.291781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c5f8637b02947eb8fa5c9447544c105'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.304397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ec224acc-73a7-4a3f-9c4a-ea619ee0c821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c5f8637b02947eb8fa5c9447544c105'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.310911 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1376/2757] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:13 2022] DELETE /identity/v3/users/9c5f8637b02947eb8fa5c9447544c105 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:13.638935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:13.640582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:13.654939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2b80f-bedb-466c-b73b-77b124331a10 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:13.737516 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:13.738196 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.738481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.738759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] PATH_INFO: `/v3/projects/12ddde56409a4ea784568b457047d05f` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.739901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ddde56409a4ea784568b457047d05f)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:13.741862 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:13.774392 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:13.810904 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ddde56409a4ea784568b457047d05f', 'request_id': 'req-641e3be8-cef8-4307-a99a-d03a3bf5de0c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.816846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ddde56409a4ea784568b457047d05f', 'request_id': 'req-641e3be8-cef8-4307-a99a-d03a3bf5de0c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.824581 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ddde56409a4ea784568b457047d05f', 'request_id': 'req-641e3be8-cef8-4307-a99a-d03a3bf5de0c'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.840674 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] The token cache is being invalidate because project 12ddde56409a4ea784568b457047d05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:13.841207 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.841939 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-641e3be8-cef8-4307-a99a-d03a3bf5de0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:13.846773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1382/2758] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:13 2022] DELETE /identity/v3/projects/12ddde56409a4ea784568b457047d05f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:13.848562 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24524321-9065-4ad2-a4a1-06db900fcd6f None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.849054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24524321-9065-4ad2-a4a1-06db900fcd6f None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.849054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24524321-9065-4ad2-a4a1-06db900fcd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.870353 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-24524321-9065-4ad2-a4a1-06db900fcd6f None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:13.871567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-24524321-9065-4ad2-a4a1-06db900fcd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:13.934740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1377/2759] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:08:13.942753 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457e08f-1c2f-409b-a042-3328013514a8 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.943050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457e08f-1c2f-409b-a042-3328013514a8 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.943335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457e08f-1c2f-409b-a042-3328013514a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.964337 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-0457e08f-1c2f-409b-a042-3328013514a8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:13.965558 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0457e08f-1c2f-409b-a042-3328013514a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:13.973597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1383/2760] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:13.977880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24524321-9065-4ad2-a4a1-06db900fcd6f None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:13.987935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24524321-9065-4ad2-a4a1-06db900fcd6f None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:13.992577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:13.993196 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.993480 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.993752 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.994430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:08:13.994738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:13.996375 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.009061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1378/2761] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:08:14.013289 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457e08f-1c2f-409b-a042-3328013514a8 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.022765 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457e08f-1c2f-409b-a042-3328013514a8 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.028092 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.028729 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.029008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.029285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.029919 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.031823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.061048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1384/2762] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:14 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:08:14.065213 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.076012 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175373-d583-4c3f-a68e-956fdeab4df6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.102293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:14.135835 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.136475 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.136825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.137019 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.137644 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.139538 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.162637 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1379/2763] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:14 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:14.163774 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.175202 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c6cd0-6853-4821-9e9a-65e959430d98 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.179846 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.180487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.180763 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.181033 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.181677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.183266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.194946 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1385/2764] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:14.198925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.209191 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32967f37-4ef1-4082-8e85-07d3cf54c887 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.214827 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda8526-7252-4908-91ea-0c988e6dd358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.215513 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8526-7252-4908-91ea-0c988e6dd358 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.215790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8526-7252-4908-91ea-0c988e6dd358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.216061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8526-7252-4908-91ea-0c988e6dd358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.216730 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda8526-7252-4908-91ea-0c988e6dd358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.218338 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda8526-7252-4908-91ea-0c988e6dd358 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.229681 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1380/2765] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:14.233702 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.243696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c65e0-d4d0-4222-b8f1-a09fe7a3ff61 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.248309 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.248954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.249235 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.249507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] PATH_INFO: `/v3/projects/5fde4be540f54c1f8df2266cae176c43/users/d5a8801437d847d09a48592ea040a043/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.263596 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fde4be540f54c1f8df2266cae176c43, user_id=d5a8801437d847d09a48592ea040a043, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.266279 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.279475 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1386/2766] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:14 2022] PUT /identity/v3/projects/5fde4be540f54c1f8df2266cae176c43/users/d5a8801437d847d09a48592ea040a043/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:14.289724 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58561a-56b0-4599-9960-d1f0b9541944 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.290015 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58561a-56b0-4599-9960-d1f0b9541944 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.290293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58561a-56b0-4599-9960-d1f0b9541944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.318387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-5f58561a-56b0-4599-9960-d1f0b9541944 None None] MFA Rules not processed for user `d5a8801437d847d09a48592ea040a043`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:14.319602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-5f58561a-56b0-4599-9960-d1f0b9541944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:14.388740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1381/2767] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:14.393189 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.454414 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa1785-1602-4d24-8767-ae400c96a9ce admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.459063 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.459668 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.459947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.460255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.461546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.463448 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.490747 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1387/2768] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:14 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:08:14.495504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58561a-56b0-4599-9960-d1f0b9541944 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.506046 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58561a-56b0-4599-9960-d1f0b9541944 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.562569 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.563188 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.563471 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.563745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.564418 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.566342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.589224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1382/2769] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:14 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:14.593318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.604400 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d36245-4212-4df9-9fb9-232df6f64e55 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.609107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.609781 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.610130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.610466 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.611193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.612916 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.626221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1388/2770] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:08:14.630339 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.646950 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83446c9d-4687-43c3-852a-74516e25b716 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.655128 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.655809 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.656089 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.656396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] PATH_INFO: `/v3/projects/746ac641caf84c2bae87ca05ccb3e6c7/users/3c2b9f99036f42f8a7310d6bf689a6d2/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.680136 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=746ac641caf84c2bae87ca05ccb3e6c7, user_id=3c2b9f99036f42f8a7310d6bf689a6d2, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.682831 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.709839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1383/2771] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:14 2022] PUT /identity/v3/projects/746ac641caf84c2bae87ca05ccb3e6c7/users/3c2b9f99036f42f8a7310d6bf689a6d2/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:14.715349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.727395 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7837c-e859-4490-a642-81eeb9062ab4 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.748987 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:14.786554 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f28d6-9748-4165-aaf4-6e0f60b5f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.787172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f28d6-9748-4165-aaf4-6e0f60b5f344 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.787455 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f28d6-9748-4165-aaf4-6e0f60b5f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.787730 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f28d6-9748-4165-aaf4-6e0f60b5f344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.788406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f28d6-9748-4165-aaf4-6e0f60b5f344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.790628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f28d6-9748-4165-aaf4-6e0f60b5f344 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.800253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1389/2772] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:14.804491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.810148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:14.816504 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd4302-3d5d-430d-980c-0263d724b3ea admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.821441 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.822094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.822377 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.822654 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] PATH_INFO: `/v3/projects/746ac641caf84c2bae87ca05ccb3e6c7/users/3c2b9f99036f42f8a7310d6bf689a6d2/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.824640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=746ac641caf84c2bae87ca05ccb3e6c7, user_id=3c2b9f99036f42f8a7310d6bf689a6d2, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.827269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.837807 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1384/2773] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:14 2022] PUT /identity/v3/projects/746ac641caf84c2bae87ca05ccb3e6c7/users/3c2b9f99036f42f8a7310d6bf689a6d2/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:14.848592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaff7c-693f-4159-93c1-1419a41b39da None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.848885 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaff7c-693f-4159-93c1-1419a41b39da None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.849161 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaff7c-693f-4159-93c1-1419a41b39da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.876595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-d4eaff7c-693f-4159-93c1-1419a41b39da None None] MFA Rules not processed for user `3c2b9f99036f42f8a7310d6bf689a6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:14.877801 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d4eaff7c-693f-4159-93c1-1419a41b39da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:14.944381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1390/2774] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:14.955086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.956880 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:15.013515 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef46b-d352-4174-9b1f-298f3c3a3afb admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:15.017197 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d80bef-a1fc-4745-9726-d997ec1a2524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:15.017818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d80bef-a1fc-4745-9726-d997ec1a2524 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:15.018092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d80bef-a1fc-4745-9726-d997ec1a2524 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:15.018361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d80bef-a1fc-4745-9726-d997ec1a2524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:15.028787 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d80bef-a1fc-4745-9726-d997ec1a2524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:15.030248 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d80bef-a1fc-4745-9726-d997ec1a2524 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:15.040786 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1385/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:16.379994 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaff7c-693f-4159-93c1-1419a41b39da None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:16.450982 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaff7c-693f-4159-93c1-1419a41b39da None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:16.455671 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:16.456339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:16.456619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:16.456894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] PATH_INFO: `/v3/users/d5a8801437d847d09a48592ea040a043` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:16.458026 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a8801437d847d09a48592ea040a043)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:16.459996 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:16.510226 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a8801437d847d09a48592ea040a043'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:16.510882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a8801437d847d09a48592ea040a043'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:16.512585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:16.520597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a8801437d847d09a48592ea040a043'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:16.523459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a8801437d847d09a48592ea040a043'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:16.535680 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a8801437d847d09a48592ea040a043'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:16.548432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1391/2776] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:16 2022] DELETE /identity/v3/users/d5a8801437d847d09a48592ea040a043 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:16.553072 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:16.553561 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:16.553698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:16.573667 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:16.574935 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:16.638540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1386/2777] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:08:16.648617 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:16.650357 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:16.711436 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eacf71-a719-4be2-988b-a7478ebde5ed admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:16.715164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:16.715796 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:16.716074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:16.716393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:16.726740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:16.728218 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:16.738594 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1392/2778] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:17.047896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:17.049362 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:17.062545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e7b42-bc99-4d03-ac90-738071c10f68 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:17.068047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:17.068899 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:17.069396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:17.069771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] PATH_INFO: `/v3/projects/5fde4be540f54c1f8df2266cae176c43` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:17.071286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fde4be540f54c1f8df2266cae176c43)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:17.073460 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:17.147315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fde4be540f54c1f8df2266cae176c43', 'request_id': 'req-a824f5cf-2c28-49ce-8ab1-06993b1e63db'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.153656 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fde4be540f54c1f8df2266cae176c43', 'request_id': 'req-a824f5cf-2c28-49ce-8ab1-06993b1e63db'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.160083 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fde4be540f54c1f8df2266cae176c43', 'request_id': 'req-a824f5cf-2c28-49ce-8ab1-06993b1e63db'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.173602 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] The token cache is being invalidate because project 5fde4be540f54c1f8df2266cae176c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:17.173895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.174606 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.176117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1387/2779] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:17 2022] DELETE /identity/v3/projects/5fde4be540f54c1f8df2266cae176c43 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:17.179984 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:17.190686 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884c390-6b81-4f48-8242-14c3f9aad21c service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:17.258666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:17.259498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:17.259779 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:17.260091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] PATH_INFO: `/v3/users/3c2b9f99036f42f8a7310d6bf689a6d2` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:17.261262 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2b9f99036f42f8a7310d6bf689a6d2)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:17.263322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:17.311046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2b9f99036f42f8a7310d6bf689a6d2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.312192 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2b9f99036f42f8a7310d6bf689a6d2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.320716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2b9f99036f42f8a7310d6bf689a6d2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.322318 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2b9f99036f42f8a7310d6bf689a6d2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.333728 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:17.335663 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-f8b643e8-85ef-445e-906e-b4466b4bfb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2b9f99036f42f8a7310d6bf689a6d2'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.355647 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1393/2780] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:17 2022] DELETE /identity/v3/users/3c2b9f99036f42f8a7310d6bf689a6d2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:17.780010 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:17.781474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:17.800582 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824f5cf-2c28-49ce-8ab1-06993b1e63db admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:17.813622 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:17.861422 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:17.862171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:17.862455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:17.862728 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] PATH_INFO: `/v3/projects/746ac641caf84c2bae87ca05ccb3e6c7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:17.863844 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=746ac641caf84c2bae87ca05ccb3e6c7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:17.865825 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:17.911817 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746ac641caf84c2bae87ca05ccb3e6c7', 'request_id': 'req-764860d7-41a2-41d3-baf5-36246d5be749'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.917286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746ac641caf84c2bae87ca05ccb3e6c7', 'request_id': 'req-764860d7-41a2-41d3-baf5-36246d5be749'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.923332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746ac641caf84c2bae87ca05ccb3e6c7', 'request_id': 'req-764860d7-41a2-41d3-baf5-36246d5be749'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.935709 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] The token cache is being invalidate because project 746ac641caf84c2bae87ca05ccb3e6c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:17.935998 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.936666 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-764860d7-41a2-41d3-baf5-36246d5be749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:17.938158 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1388/2781] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:17 2022] DELETE /identity/v3/projects/746ac641caf84c2bae87ca05ccb3e6c7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:17.943064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba185fd-8eb6-467b-baf4-5c020db52202 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:17.943583 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba185fd-8eb6-467b-baf4-5c020db52202 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:17.943691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba185fd-8eb6-467b-baf4-5c020db52202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:17.964803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-cba185fd-8eb6-467b-baf4-5c020db52202 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:17.966029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-cba185fd-8eb6-467b-baf4-5c020db52202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:18.024794 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1394/2782] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:18.032571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9d74c-1aa9-4a86-9758-e362913678bd None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.032859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9d74c-1aa9-4a86-9758-e362913678bd None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.033130 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9d74c-1aa9-4a86-9758-e362913678bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.052634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-10e9d74c-1aa9-4a86-9758-e362913678bd None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:18.053893 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-10e9d74c-1aa9-4a86-9758-e362913678bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:18.062529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1389/2783] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:18.066660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba185fd-8eb6-467b-baf4-5c020db52202 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.076751 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba185fd-8eb6-467b-baf4-5c020db52202 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.081409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.082013 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.082296 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.082571 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.083215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:08:18.083518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.085148 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.096697 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1395/2784] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:08:18.100818 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9d74c-1aa9-4a86-9758-e362913678bd None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.111254 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9d74c-1aa9-4a86-9758-e362913678bd None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.116103 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.116734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.117540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.117811 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.118440 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.120389 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.147987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1390/2785] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:18 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:18.152069 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.162336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639e7f-504d-48fe-8f13-9085c337cfeb admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.222789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.223678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.223678 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.223928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.224685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.227585 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.252514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1396/2786] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:18 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:18.253738 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.264463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef6aa0-ab5f-406d-af4f-021e79049d4e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.269168 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.269776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.270185 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.270616 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.271269 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.272910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.283468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1391/2787] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:18.289754 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.299814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd908a2a-89fe-42ac-a104-13b9d125d7fa admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.305176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbfc0e-a970-4301-8271-31d8b8ccde64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.305824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbfc0e-a970-4301-8271-31d8b8ccde64 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.306100 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbfc0e-a970-4301-8271-31d8b8ccde64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.306367 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbfc0e-a970-4301-8271-31d8b8ccde64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.307002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbfc0e-a970-4301-8271-31d8b8ccde64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.308638 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbfc0e-a970-4301-8271-31d8b8ccde64 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.318479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1397/2788] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:18.322812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.332740 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018537dd-ba6a-4288-9ce2-cd8ec3a5a1d8 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.337675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.338315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.338591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.338863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] PATH_INFO: `/v3/projects/5754f29b91d64ce3843d6c8768961c1d/users/1a2d5ba74c9a4096941ebfe53a74b9fe/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.354245 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5754f29b91d64ce3843d6c8768961c1d, user_id=1a2d5ba74c9a4096941ebfe53a74b9fe, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.356982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.370313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1392/2789] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:18 2022] PUT /identity/v3/projects/5754f29b91d64ce3843d6c8768961c1d/users/1a2d5ba74c9a4096941ebfe53a74b9fe/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:18.380422 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e3a29-f392-4253-9117-30b6e9d522c1 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.380709 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e3a29-f392-4253-9117-30b6e9d522c1 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.380980 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e3a29-f392-4253-9117-30b6e9d522c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.408911 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-586e3a29-f392-4253-9117-30b6e9d522c1 None None] MFA Rules not processed for user `1a2d5ba74c9a4096941ebfe53a74b9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:18.410128 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-586e3a29-f392-4253-9117-30b6e9d522c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:18.452892 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:18.480043 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1398/2790] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:18.484574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.530466 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:18.549335 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31986dfe-2ff9-41e9-a8fc-d5ac75041a62 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.553937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.555029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.555304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.555570 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.556261 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.558142 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.583961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1393/2791] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:18 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:18.588335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e3a29-f392-4253-9117-30b6e9d522c1 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.598676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e3a29-f392-4253-9117-30b6e9d522c1 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.652592 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.653195 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.653473 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.653743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.654365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.656285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.676917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1399/2792] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:18 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:18.681361 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.691873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced72d29-7d40-41f0-9f71-e1cd250ad547 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.697204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.697806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.698087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.698364 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.699009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.700757 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.711442 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1394/2793] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:18.715485 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.725249 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610feb5-b9b2-4907-957a-fb20734c74db admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.729896 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.730529 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.730811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.731082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] PATH_INFO: `/v3/projects/2d7aceaba0834b459ac9ae7ca7c2d333/users/d5b1cbe8bdaa4a199b1f3f25cb142138/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.745860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7aceaba0834b459ac9ae7ca7c2d333, user_id=d5b1cbe8bdaa4a199b1f3f25cb142138, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.748504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.761331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1400/2794] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:18 2022] PUT /identity/v3/projects/2d7aceaba0834b459ac9ae7ca7c2d333/users/d5b1cbe8bdaa4a199b1f3f25cb142138/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:18.765154 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.775098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948acda2-ff55-4221-874b-36f5650dd0f4 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.830547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bf7d6-e678-451f-9195-0c29c8db4267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.831428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf7d6-e678-451f-9195-0c29c8db4267 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.831428 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf7d6-e678-451f-9195-0c29c8db4267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.831734 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf7d6-e678-451f-9195-0c29c8db4267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.832470 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bf7d6-e678-451f-9195-0c29c8db4267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.834059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bf7d6-e678-451f-9195-0c29c8db4267 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.844325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1395/2795] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:18.848408 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:18.858537 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26ba01-ecd7-412c-aa01-33cae8a543e1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:18.863136 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:18.863772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.864050 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.864426 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] PATH_INFO: `/v3/projects/2d7aceaba0834b459ac9ae7ca7c2d333/users/d5b1cbe8bdaa4a199b1f3f25cb142138/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.866614 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7aceaba0834b459ac9ae7ca7c2d333, user_id=d5b1cbe8bdaa4a199b1f3f25cb142138, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:18.869510 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:18.883549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1401/2796] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:18 2022] PUT /identity/v3/projects/2d7aceaba0834b459ac9ae7ca7c2d333/users/d5b1cbe8bdaa4a199b1f3f25cb142138/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:18.894991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:18.895354 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:18.895578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:18.930313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] MFA Rules not processed for user `d5b1cbe8bdaa4a199b1f3f25cb142138`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:18.931599 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:19.031009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1396/2797] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:08:19.042553 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:19.044424 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:19.100217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3ef5-a6a4-420f-bb5c-5f56737457a2 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:19.104543 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:19.105089 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:19.105341 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:19.105607 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:19.114547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:19.115971 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:19.125793 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1402/2798] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:19.169727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:19.171111 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:19.183722 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145be44-578b-4fad-ae02-156a95de4f58 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:19.187271 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a5f11-52e2-4b48-9174-a97dd9fd259c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:19.187980 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5f11-52e2-4b48-9174-a97dd9fd259c service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:19.188309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5f11-52e2-4b48-9174-a97dd9fd259c service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:19.188587 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5f11-52e2-4b48-9174-a97dd9fd259c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:19.221413 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:19.248126 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a5f11-52e2-4b48-9174-a97dd9fd259c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:19.249662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a5f11-52e2-4b48-9174-a97dd9fd259c service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:19.258478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1397/2799] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:22.752110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:22.792517 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:22.814074 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0118e8-0080-47a2-8eff-7f1ae2dfcb89 service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:22.818669 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:22.819301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:22.819582 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:22.820396 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] PATH_INFO: `/v3/users/1a2d5ba74c9a4096941ebfe53a74b9fe` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:22.821683 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2d5ba74c9a4096941ebfe53a74b9fe)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:22.823631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:22.877108 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2d5ba74c9a4096941ebfe53a74b9fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:22.877824 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2d5ba74c9a4096941ebfe53a74b9fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:22.887143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2d5ba74c9a4096941ebfe53a74b9fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:22.888169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2d5ba74c9a4096941ebfe53a74b9fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:22.905743 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2d5ba74c9a4096941ebfe53a74b9fe'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:22.920978 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1403/2800] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:22 2022] DELETE /identity/v3/users/1a2d5ba74c9a4096941ebfe53a74b9fe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:22.925772 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:22.926293 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:22.926456 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:23.071283 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:23.072455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:23.130359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1398/2801] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:23.140642 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:23.142029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:23.197942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b1251-3fe2-496b-8373-d36162fe373e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:23.201546 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:23.202147 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:23.202425 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:23.202696 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:23.211681 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:23.213139 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:23.222413 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1404/2802] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:23.490365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:23.492499 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:23.504611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8dae83-d6a1-4f81-8bd1-2bfadea4bd18 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:23.509280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:23.509919 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:23.510247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:23.510462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] PATH_INFO: `/v3/projects/5754f29b91d64ce3843d6c8768961c1d` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:23.511854 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] RBAC: Authorizing `identity:delete_project(project_id=5754f29b91d64ce3843d6c8768961c1d)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:23.514316 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:23.562398 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5754f29b91d64ce3843d6c8768961c1d', 'request_id': 'req-fe82c13d-4a4a-4286-acba-2e734008cace'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.567937 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5754f29b91d64ce3843d6c8768961c1d', 'request_id': 'req-fe82c13d-4a4a-4286-acba-2e734008cace'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.574057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5754f29b91d64ce3843d6c8768961c1d', 'request_id': 'req-fe82c13d-4a4a-4286-acba-2e734008cace'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.586860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] The token cache is being invalidate because project 5754f29b91d64ce3843d6c8768961c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:23.587149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.587777 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.589279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1399/2803] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:23 2022] DELETE /identity/v3/projects/5754f29b91d64ce3843d6c8768961c1d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:23.593233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:23.603283 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf5728-b15b-4623-a795-45883fb52739 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:23.660492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:23.661129 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:23.661437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:23.661712 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] PATH_INFO: `/v3/users/d5b1cbe8bdaa4a199b1f3f25cb142138` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:23.662854 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b1cbe8bdaa4a199b1f3f25cb142138)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:23.665317 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:23.666771 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:23.712975 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b1cbe8bdaa4a199b1f3f25cb142138'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.713619 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b1cbe8bdaa4a199b1f3f25cb142138'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.720636 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b1cbe8bdaa4a199b1f3f25cb142138'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.721243 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b1cbe8bdaa4a199b1f3f25cb142138'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.733690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-57e2d6c2-0c39-43fb-a191-9d7a0ddbe263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b1cbe8bdaa4a199b1f3f25cb142138'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:23.746039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1405/2804] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:23 2022] DELETE /identity/v3/users/d5b1cbe8bdaa4a199b1f3f25cb142138 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:24.012611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.014074 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:24.029087 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82c13d-4a4a-4286-acba-2e734008cace admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.094770 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.095408 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.095682 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.095949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] PATH_INFO: `/v3/projects/2d7aceaba0834b459ac9ae7ca7c2d333` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.097098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d7aceaba0834b459ac9ae7ca7c2d333)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.098978 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.100696 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:24.150524 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d7aceaba0834b459ac9ae7ca7c2d333', 'request_id': 'req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:24.156640 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d7aceaba0834b459ac9ae7ca7c2d333', 'request_id': 'req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:24.163390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d7aceaba0834b459ac9ae7ca7c2d333', 'request_id': 'req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:24.176025 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] The token cache is being invalidate because project 2d7aceaba0834b459ac9ae7ca7c2d333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:24.176419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:24.177205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-ba6efafa-6c6b-4e9f-97a4-7c4d308df6bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:24.178986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1400/2805] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:24 2022] DELETE /identity/v3/projects/2d7aceaba0834b459ac9ae7ca7c2d333 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:24.183770 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71276f71-8a20-4e9a-9012-af6c1c06ae13 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.184240 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71276f71-8a20-4e9a-9012-af6c1c06ae13 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.184353 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71276f71-8a20-4e9a-9012-af6c1c06ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.205039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-71276f71-8a20-4e9a-9012-af6c1c06ae13 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:24.206244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-71276f71-8a20-4e9a-9012-af6c1c06ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:24.267959 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1406/2806] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:24.276257 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9e003-c4ab-4147-9856-6d0fb5129739 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.276621 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9e003-c4ab-4147-9856-6d0fb5129739 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.276898 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9e003-c4ab-4147-9856-6d0fb5129739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.297118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-80a9e003-c4ab-4147-9856-6d0fb5129739 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:24.298343 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-80a9e003-c4ab-4147-9856-6d0fb5129739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:24.306444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1401/2807] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:24.310932 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71276f71-8a20-4e9a-9012-af6c1c06ae13 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.321185 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71276f71-8a20-4e9a-9012-af6c1c06ae13 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.325945 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.326551 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.326829 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.327099 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.327782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:08:24.328083 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.329742 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.341962 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1407/2808] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:08:24.346397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9e003-c4ab-4147-9856-6d0fb5129739 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.356226 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9e003-c4ab-4147-9856-6d0fb5129739 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.361129 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.361763 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.362039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.362304 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.362929 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.364890 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.395450 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1402/2809] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:24.399203 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.409882 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7152d3-9da8-4356-a882-79df45b9eb6d admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.478966 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.479577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.479857 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.480130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.480789 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.482705 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.494490 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:24.508521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1408/2810] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:08:24.512647 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.523563 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6917-c2f4-44a5-8bba-15cbe19b26ca admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.528262 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.528867 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.529147 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.529420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.530067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.531668 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.543506 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1403/2811] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:24.547536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.557903 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684cb9e-535c-4912-a165-119c23d2c988 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.562848 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140af00-95b6-4942-aa87-e9ab5c5e620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.563532 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140af00-95b6-4942-aa87-e9ab5c5e620a admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.563905 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140af00-95b6-4942-aa87-e9ab5c5e620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.564294 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140af00-95b6-4942-aa87-e9ab5c5e620a admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.565058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140af00-95b6-4942-aa87-e9ab5c5e620a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.566791 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140af00-95b6-4942-aa87-e9ab5c5e620a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.580690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1409/2812] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:24.584896 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.596813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf2dd-f31b-4529-ace8-684f111fe176 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.602881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.604277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.604776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.605341 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] PATH_INFO: `/v3/projects/8880db0a26e347869135c6076ab603bd/users/4316fecdb8ad48a5bbb67a676b6e5fe9/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.623541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8880db0a26e347869135c6076ab603bd, user_id=4316fecdb8ad48a5bbb67a676b6e5fe9, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.626396 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.640200 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1404/2813] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:24 2022] PUT /identity/v3/projects/8880db0a26e347869135c6076ab603bd/users/4316fecdb8ad48a5bbb67a676b6e5fe9/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:24.707297 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8f28-de08-4934-9657-c311fd9f637d None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.707580 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8f28-de08-4934-9657-c311fd9f637d None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.707858 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8f28-de08-4934-9657-c311fd9f637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.735906 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-f67e8f28-de08-4934-9657-c311fd9f637d None None] MFA Rules not processed for user `4316fecdb8ad48a5bbb67a676b6e5fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:24.737169 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-f67e8f28-de08-4934-9657-c311fd9f637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:24.800577 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1410/2814] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:24.805035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.865514 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61701aa-7cb1-44e5-973d-24e45f046d18 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.870392 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.870994 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.871272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.871545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.872230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.874116 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.901042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1405/2815] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:24.905199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e8f28-de08-4934-9657-c311fd9f637d None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.916215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e8f28-de08-4934-9657-c311fd9f637d None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.973320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.973935 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.974212 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.974478 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.975107 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.977060 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.997153 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1411/2816] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:08:25.001091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:25.011691 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ce1d1-e49c-4748-98a8-6be29a8443aa admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:25.016229 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:25.016823 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.017085 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.017352 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.017987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:25.019567 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:25.025332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:25.030791 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1406/2817] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:25.035345 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:25.045441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f406b-a7e6-441c-94a7-0200fc0590b7 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:25.050326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:25.051023 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.051303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.051598 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] PATH_INFO: `/v3/projects/9c1958c6a2a94651b8fe71d8b1221cf8/users/35d0efc3db6649ee9011ee8eda8295d3/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.066316 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1958c6a2a94651b8fe71d8b1221cf8, user_id=35d0efc3db6649ee9011ee8eda8295d3, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:25.069011 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:25.082703 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1412/2818] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:25 2022] PUT /identity/v3/projects/9c1958c6a2a94651b8fe71d8b1221cf8/users/35d0efc3db6649ee9011ee8eda8295d3/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:25.087363 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:25.097387 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29473ec6-daed-4c4e-a25d-007b3da0d6db admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:25.153113 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d1487-0c59-458a-b2c7-178b57139af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:25.153707 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d1487-0c59-458a-b2c7-178b57139af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.153983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d1487-0c59-458a-b2c7-178b57139af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.154251 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d1487-0c59-458a-b2c7-178b57139af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.154874 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d1487-0c59-458a-b2c7-178b57139af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:25.156478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d1487-0c59-458a-b2c7-178b57139af3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:25.165983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1407/2819] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:25.170231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:25.180386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d83ba0-2c84-4aed-ba5c-a23b3b47bb86 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:25.184957 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:25.185599 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.185873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.186140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] PATH_INFO: `/v3/projects/9c1958c6a2a94651b8fe71d8b1221cf8/users/35d0efc3db6649ee9011ee8eda8295d3/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.188065 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1958c6a2a94651b8fe71d8b1221cf8, user_id=35d0efc3db6649ee9011ee8eda8295d3, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:25.190704 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:25.201695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1413/2820] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:25 2022] PUT /identity/v3/projects/9c1958c6a2a94651b8fe71d8b1221cf8/users/35d0efc3db6649ee9011ee8eda8295d3/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:25.212358 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d376-7f13-417f-baa2-83c6f1cffde6 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.212706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d376-7f13-417f-baa2-83c6f1cffde6 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.212987 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d376-7f13-417f-baa2-83c6f1cffde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.240429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-fac2d376-7f13-417f-baa2-83c6f1cffde6 None None] MFA Rules not processed for user `35d0efc3db6649ee9011ee8eda8295d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:25.241648 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-fac2d376-7f13-417f-baa2-83c6f1cffde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:25.312436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1408/2821] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:25.323684 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:25.325525 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:25.364459 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:25.388250 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537f3b-83fa-455f-b58d-1a0cc077da80 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:25.391886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6546c78-9ecc-46a3-8d74-98c5faf0aeb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:25.392536 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6546c78-9ecc-46a3-8d74-98c5faf0aeb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.392813 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6546c78-9ecc-46a3-8d74-98c5faf0aeb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.393082 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6546c78-9ecc-46a3-8d74-98c5faf0aeb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.402282 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6546c78-9ecc-46a3-8d74-98c5faf0aeb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:25.403695 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6546c78-9ecc-46a3-8d74-98c5faf0aeb4 service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:25.413106 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1414/2822] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:25.548826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d376-7f13-417f-baa2-83c6f1cffde6 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:25.610577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d376-7f13-417f-baa2-83c6f1cffde6 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:25.614922 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:25.615544 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.615819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.616086 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] PATH_INFO: `/v3/users/4316fecdb8ad48a5bbb67a676b6e5fe9` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.617246 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4316fecdb8ad48a5bbb67a676b6e5fe9)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:25.619179 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:25.641281 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:25.665974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4316fecdb8ad48a5bbb67a676b6e5fe9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:25.672710 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4316fecdb8ad48a5bbb67a676b6e5fe9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:25.679992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4316fecdb8ad48a5bbb67a676b6e5fe9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:25.680491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4316fecdb8ad48a5bbb67a676b6e5fe9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:25.691982 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4316fecdb8ad48a5bbb67a676b6e5fe9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:25.698216 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1409/2823] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:25 2022] DELETE /identity/v3/users/4316fecdb8ad48a5bbb67a676b6e5fe9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:25.702708 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.702992 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.703266 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.722197 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:25.723380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:25.780380 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1415/2824] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:25.790102 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:25.791501 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:25.847843 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ed12b-84fa-42ec-8f1e-b59c9b449be6 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:25.851516 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:25.852118 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:25.852429 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:25.852699 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:25.861634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:25.863062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:25.872607 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1410/2825] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:26.133547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:26.135277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:26.157660 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1befc3b-866f-4b0f-a17c-eaaab8929846 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:26.163378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:26.164827 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:26.165179 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:26.165527 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] PATH_INFO: `/v3/projects/8880db0a26e347869135c6076ab603bd` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:26.166749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8880db0a26e347869135c6076ab603bd)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:26.168787 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:26.216841 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8880db0a26e347869135c6076ab603bd', 'request_id': 'req-028e4fd4-24fb-4382-b43c-80126dac0210'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.222291 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8880db0a26e347869135c6076ab603bd', 'request_id': 'req-028e4fd4-24fb-4382-b43c-80126dac0210'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.229130 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8880db0a26e347869135c6076ab603bd', 'request_id': 'req-028e4fd4-24fb-4382-b43c-80126dac0210'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.241888 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] The token cache is being invalidate because project 8880db0a26e347869135c6076ab603bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:26.242176 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.242807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.244326 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1416/2826] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:26 2022] DELETE /identity/v3/projects/8880db0a26e347869135c6076ab603bd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:26.248277 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:26.258217 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db79db-f17a-408d-bd91-967723e82af2 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:26.314040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:26.314664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:26.314943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:26.315214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] PATH_INFO: `/v3/users/35d0efc3db6649ee9011ee8eda8295d3` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:26.316360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d0efc3db6649ee9011ee8eda8295d3)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:26.318286 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:26.362021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:26.363877 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d0efc3db6649ee9011ee8eda8295d3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.373386 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d0efc3db6649ee9011ee8eda8295d3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.381601 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d0efc3db6649ee9011ee8eda8295d3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.382806 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d0efc3db6649ee9011ee8eda8295d3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.397634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b9c3e249-311b-4a86-93c0-f4520f44271b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d0efc3db6649ee9011ee8eda8295d3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.404432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1411/2827] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:26 2022] DELETE /identity/v3/users/35d0efc3db6649ee9011ee8eda8295d3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:26.683664 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:26.685305 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:26.700805 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e4fd4-24fb-4382-b43c-80126dac0210 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:26.755780 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:26.766277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:26.767143 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:26.767782 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:26.768221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] PATH_INFO: `/v3/projects/9c1958c6a2a94651b8fe71d8b1221cf8` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:26.769625 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1958c6a2a94651b8fe71d8b1221cf8)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:26.772036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:26.819338 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1958c6a2a94651b8fe71d8b1221cf8', 'request_id': 'req-774c7300-3c12-4950-8418-ed076236f115'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.826776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1958c6a2a94651b8fe71d8b1221cf8', 'request_id': 'req-774c7300-3c12-4950-8418-ed076236f115'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.833103 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1958c6a2a94651b8fe71d8b1221cf8', 'request_id': 'req-774c7300-3c12-4950-8418-ed076236f115'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.845531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] The token cache is being invalidate because project 9c1958c6a2a94651b8fe71d8b1221cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:26.845822 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.846437 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-774c7300-3c12-4950-8418-ed076236f115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:26.847926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1417/2828] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:26 2022] DELETE /identity/v3/projects/9c1958c6a2a94651b8fe71d8b1221cf8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:26.852863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd11d-b6f9-4baa-9a42-9fd994864ab8 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:26.853155 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd11d-b6f9-4baa-9a42-9fd994864ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:26.853436 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd11d-b6f9-4baa-9a42-9fd994864ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:26.873220 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1bdfd11d-b6f9-4baa-9a42-9fd994864ab8 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:26.874414 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1bdfd11d-b6f9-4baa-9a42-9fd994864ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:26.932266 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1412/2829] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:26.940038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751081e-35e7-4895-88dd-4f9a49113886 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:26.940376 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751081e-35e7-4895-88dd-4f9a49113886 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:26.940653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751081e-35e7-4895-88dd-4f9a49113886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:26.967073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-3751081e-35e7-4895-88dd-4f9a49113886 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:26.968339 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3751081e-35e7-4895-88dd-4f9a49113886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:26.976217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1418/2830] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:26.980617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd11d-b6f9-4baa-9a42-9fd994864ab8 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:26.997278 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd11d-b6f9-4baa-9a42-9fd994864ab8 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.001928 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.002541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.002820 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.003093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] PATH_INFO: `/v3/domains` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.003746 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:08:27.004049 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.005683 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.017966 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1413/2831] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:08:27.022215 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751081e-35e7-4895-88dd-4f9a49113886 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.031685 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751081e-35e7-4895-88dd-4f9a49113886 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.036397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.036997 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.037277 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.037550 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.038170 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.040278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.067019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1419/2832] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:27 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:27.071839 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.082017 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901f1cf-1492-4cd6-8cc6-fb9261cad533 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.138856 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.139462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.139761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.140034 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.140686 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.142589 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.162292 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1414/2833] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:27 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:27.167312 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.177894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3f1d-2354-42be-970b-b516e18f0478 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.183302 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.183936 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.184255 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.184541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.185184 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.186776 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.196690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1420/2834] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:08:27.200558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.210307 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df0236-9323-4be1-bd11-e32d7265b5ae admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.215215 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d6c18-0fc2-4871-b467-5a2bb9c0644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.215805 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d6c18-0fc2-4871-b467-5a2bb9c0644c admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.216092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d6c18-0fc2-4871-b467-5a2bb9c0644c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.216402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d6c18-0fc2-4871-b467-5a2bb9c0644c admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.217038 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d6c18-0fc2-4871-b467-5a2bb9c0644c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.218664 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d6c18-0fc2-4871-b467-5a2bb9c0644c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.229659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1415/2835] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:27.233650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.244462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658aac9-0e25-4301-8444-67d92235f7c3 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.249389 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.250105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.250387 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.250666 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] PATH_INFO: `/v3/projects/bfba629449364f7dad5f430b083419db/users/0c31c305aa044850add91ce088db9eca/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.265045 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfba629449364f7dad5f430b083419db, user_id=0c31c305aa044850add91ce088db9eca, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.267687 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.280125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1421/2836] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:27 2022] PUT /identity/v3/projects/bfba629449364f7dad5f430b083419db/users/0c31c305aa044850add91ce088db9eca/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:27.292054 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da91ce-a209-4f95-9ec2-3600b9001795 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.292614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da91ce-a209-4f95-9ec2-3600b9001795 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.292614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da91ce-a209-4f95-9ec2-3600b9001795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.322822 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-81da91ce-a209-4f95-9ec2-3600b9001795 None None] MFA Rules not processed for user `0c31c305aa044850add91ce088db9eca`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:27.324042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-81da91ce-a209-4f95-9ec2-3600b9001795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:27.341033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:27.390195 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1416/2837] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:27.394814 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.456714 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a2b4-21b2-4b65-972e-5dbcdec17bac admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.461675 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.462275 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.462552 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.462823 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] PATH_INFO: `/v3/projects` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.463468 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.465416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.490942 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1422/2838] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:27 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:27.495712 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da91ce-a209-4f95-9ec2-3600b9001795 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.506333 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da91ce-a209-4f95-9ec2-3600b9001795 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.563700 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.564337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.564611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.565004 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] PATH_INFO: `/v3/users` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.565491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.567390 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.594011 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1417/2839] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:27 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:08:27.595410 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.602371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:27.606556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceaeac-4246-4d95-ac39-29a70d62f70e admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.612756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.613798 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.614217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.614557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.615281 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.618125 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.629323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1423/2840] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:27.633642 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.644715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db53b-5e3f-4ba0-b597-f9ccc347884e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.650094 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.650794 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.651092 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.651370 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] PATH_INFO: `/v3/projects/e64549bef4a1477bb34dcd55c0f0eb05/users/32aa366292df4c9a970f929482d63035/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.668508 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64549bef4a1477bb34dcd55c0f0eb05, user_id=32aa366292df4c9a970f929482d63035, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.671183 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.685650 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1418/2841] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:27 2022] PUT /identity/v3/projects/e64549bef4a1477bb34dcd55c0f0eb05/users/32aa366292df4c9a970f929482d63035/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:27.689651 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.700518 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f7288-3bdf-461d-982b-67c78aa69532 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.765495 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feccf8-ca16-4cd5-a187-2ebe8214b08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.766111 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feccf8-ca16-4cd5-a187-2ebe8214b08b admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.766391 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feccf8-ca16-4cd5-a187-2ebe8214b08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.766662 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feccf8-ca16-4cd5-a187-2ebe8214b08b admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.767307 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98feccf8-ca16-4cd5-a187-2ebe8214b08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.769035 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98feccf8-ca16-4cd5-a187-2ebe8214b08b admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.780024 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1424/2842] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:27.784332 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.794907 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894626d0-bd9f-4ab5-84f9-7028546eed2e admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.800717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.801315 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.801591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.801863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] PATH_INFO: `/v3/projects/e64549bef4a1477bb34dcd55c0f0eb05/users/32aa366292df4c9a970f929482d63035/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.803848 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64549bef4a1477bb34dcd55c0f0eb05, user_id=32aa366292df4c9a970f929482d63035, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.807036 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.817895 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1419/2843] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:27 2022] PUT /identity/v3/projects/e64549bef4a1477bb34dcd55c0f0eb05/users/32aa366292df4c9a970f929482d63035/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:27.828114 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2a9b3-7644-43e1-98e9-a45c7d837148 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.828541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2a9b3-7644-43e1-98e9-a45c7d837148 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.828894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2a9b3-7644-43e1-98e9-a45c7d837148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.858025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-38d2a9b3-7644-43e1-98e9-a45c7d837148 None None] MFA Rules not processed for user `32aa366292df4c9a970f929482d63035`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:27.859229 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-38d2a9b3-7644-43e1-98e9-a45c7d837148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:27.934713 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1425/2844] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:27.946836 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.948983 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:28.009540 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03919b5-ecf0-4bd3-85a8-295b8deb636f admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:28.013182 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a831ad1-1a7b-4ae3-b638-d131b2178090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:28.013785 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a831ad1-1a7b-4ae3-b638-d131b2178090 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:28.014061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a831ad1-1a7b-4ae3-b638-d131b2178090 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:28.014328 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a831ad1-1a7b-4ae3-b638-d131b2178090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:28.023230 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a831ad1-1a7b-4ae3-b638-d131b2178090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:28.024813 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a831ad1-1a7b-4ae3-b638-d131b2178090 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:28.034575 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1420/2845] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:28.613300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2a9b3-7644-43e1-98e9-a45c7d837148 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:28.680360 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2a9b3-7644-43e1-98e9-a45c7d837148 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:28.684915 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:28.685538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:28.685817 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:28.686086 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] PATH_INFO: `/v3/users/0c31c305aa044850add91ce088db9eca` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:28.687310 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c31c305aa044850add91ce088db9eca)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:28.689259 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:28.706386 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:28.738016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c31c305aa044850add91ce088db9eca'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:28.738673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c31c305aa044850add91ce088db9eca'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:28.745937 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c31c305aa044850add91ce088db9eca'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:28.746557 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c31c305aa044850add91ce088db9eca'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:28.758224 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c31c305aa044850add91ce088db9eca'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:28.770014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1426/2846] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:28 2022] DELETE /identity/v3/users/0c31c305aa044850add91ce088db9eca => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:28.774490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:28.774961 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:28.775062 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:28.793933 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:28.795120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:28.802992 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:28.854018 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1421/2847] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:28.863621 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:28.865046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:28.921637 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c2736-5bfe-4fff-af3f-a24bb2fcf154 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:28.925300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:28.925886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:28.926163 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:28.926431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:28.936002 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:28.938476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:28.948081 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1427/2848] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:29.216039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:29.217537 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:29.231579 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c334b02-60f6-42d4-83c6-62365227ee0b None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:29.236496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:29.237204 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:29.237481 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:29.237755 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] PATH_INFO: `/v3/projects/bfba629449364f7dad5f430b083419db` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:29.239095 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfba629449364f7dad5f430b083419db)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:29.240951 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:29.286205 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfba629449364f7dad5f430b083419db', 'request_id': 'req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.292533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfba629449364f7dad5f430b083419db', 'request_id': 'req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.298849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfba629449364f7dad5f430b083419db', 'request_id': 'req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.311490 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] The token cache is being invalidate because project bfba629449364f7dad5f430b083419db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:29.311782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.312438 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.313916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1422/2849] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:29 2022] DELETE /identity/v3/projects/bfba629449364f7dad5f430b083419db => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:29.317872 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:29.327654 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f78788-aced-4431-a180-b5a8159557e4 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:29.385927 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:29.386559 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:29.386842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:29.387119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] PATH_INFO: `/v3/users/32aa366292df4c9a970f929482d63035` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:29.388331 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] RBAC: Authorizing `identity:delete_user(user_id=32aa366292df4c9a970f929482d63035)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:29.390206 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:29.433689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:29.441335 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32aa366292df4c9a970f929482d63035'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.442135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32aa366292df4c9a970f929482d63035'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.450158 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32aa366292df4c9a970f929482d63035'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.450738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32aa366292df4c9a970f929482d63035'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.463167 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-10b404ad-102f-400f-83fe-3239eac9377f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32aa366292df4c9a970f929482d63035'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.475140 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1428/2850] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:29 2022] DELETE /identity/v3/users/32aa366292df4c9a970f929482d63035 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:29.751660 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:29.753944 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:29.774793 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9229d57-3aa9-48ea-8bf9-946bc2729bb9 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:29.833808 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:29.834496 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:29.834704 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:29.834981 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] PATH_INFO: `/v3/projects/e64549bef4a1477bb34dcd55c0f0eb05` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:29.836419 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64549bef4a1477bb34dcd55c0f0eb05)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:29.838380 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:29.880425 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:29.890577 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64549bef4a1477bb34dcd55c0f0eb05', 'request_id': 'req-741f5971-c906-4335-811d-254ae11fc14f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.896427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64549bef4a1477bb34dcd55c0f0eb05', 'request_id': 'req-741f5971-c906-4335-811d-254ae11fc14f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.902665 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64549bef4a1477bb34dcd55c0f0eb05', 'request_id': 'req-741f5971-c906-4335-811d-254ae11fc14f'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.914999 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] The token cache is being invalidate because project e64549bef4a1477bb34dcd55c0f0eb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:29.915290 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.916101 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-741f5971-c906-4335-811d-254ae11fc14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:29.918040 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1423/2851] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:29 2022] DELETE /identity/v3/projects/e64549bef4a1477bb34dcd55c0f0eb05 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:29.922909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ab993-81a8-4a9e-8ebb-070fea6c3891 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:29.923407 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ab993-81a8-4a9e-8ebb-070fea6c3891 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:29.923541 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ab993-81a8-4a9e-8ebb-070fea6c3891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:29.944301 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c61ab993-81a8-4a9e-8ebb-070fea6c3891 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:29.946073 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c61ab993-81a8-4a9e-8ebb-070fea6c3891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:30.013873 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1429/2852] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:30.021551 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab701fb-4cf0-4469-9b6a-5211091a0f61 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.021845 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab701fb-4cf0-4469-9b6a-5211091a0f61 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.022120 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab701fb-4cf0-4469-9b6a-5211091a0f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.042047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-1ab701fb-4cf0-4469-9b6a-5211091a0f61 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:30.043270 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1ab701fb-4cf0-4469-9b6a-5211091a0f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:30.051694 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1424/2853] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:30.056899 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ab993-81a8-4a9e-8ebb-070fea6c3891 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:30.067346 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ab993-81a8-4a9e-8ebb-070fea6c3891 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:30.073019 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:30.073655 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.073938 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.074217 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.074860 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:08:30.075164 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:30.076804 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:30.092048 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1430/2854] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:08:30.098555 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab701fb-4cf0-4469-9b6a-5211091a0f61 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:30.108171 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab701fb-4cf0-4469-9b6a-5211091a0f61 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:30.113212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:30.113821 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.114104 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.114383 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.115007 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:30.117474 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:30.148558 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1425/2855] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:30 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:30.152734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:30.164416 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a56e90-afec-4816-a392-a9fa5d91929f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:30.221969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:30.222673 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.222847 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.223121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.223736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:30.225648 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:30.251245 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1431/2856] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:30 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:08:30.255342 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:30.265732 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af484d-817d-4cec-a570-c8df56fc758c admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:30.270345 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:30.270943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.271224 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.271497 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.272169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:30.273798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:30.287198 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1426/2857] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:30.289036 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:30.299025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12709af-7700-41f5-8d4e-7d52c8228c0a admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:30.303741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828e98-9fe2-4234-b570-b7cb87f11cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:30.305003 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828e98-9fe2-4234-b570-b7cb87f11cca admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.305280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828e98-9fe2-4234-b570-b7cb87f11cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.305548 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828e98-9fe2-4234-b570-b7cb87f11cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.306193 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828e98-9fe2-4234-b570-b7cb87f11cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:30.307775 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828e98-9fe2-4234-b570-b7cb87f11cca admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:30.318773 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1432/2858] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:08:30.324057 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:30.334552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f85747-897b-4b34-ae88-794ab4033144 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:30.339652 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:30.340337 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.340625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.340889 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] PATH_INFO: `/v3/projects/33ea4b4ef78f4f6ab669f62892410e3a/users/53d54e9ab4c74524bf6dd7c86609171e/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.355991 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ea4b4ef78f4f6ab669f62892410e3a, user_id=53d54e9ab4c74524bf6dd7c86609171e, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:30.358657 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:30.371773 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1427/2859] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:30 2022] PUT /identity/v3/projects/33ea4b4ef78f4f6ab669f62892410e3a/users/53d54e9ab4c74524bf6dd7c86609171e/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:30.376766 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298160a-0d85-4cdf-ba0f-7314f57452ef None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.377057 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298160a-0d85-4cdf-ba0f-7314f57452ef None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.377336 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298160a-0d85-4cdf-ba0f-7314f57452ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.395676 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-7298160a-0d85-4cdf-ba0f-7314f57452ef None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:30.396902 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7298160a-0d85-4cdf-ba0f-7314f57452ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:30.415735 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:30.461253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1433/2860] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:30.470882 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:30.472610 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:30.529420 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ecfbb-e8ee-4ff0-94bf-18dac6d2a8da admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:30.533047 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:30.533646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:30.533924 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:30.534211 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:30.544614 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:30.546044 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:30.555594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1428/2861] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:31.196254 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b22c9-68de-472c-b607-347e5ad09c1f None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:31.196724 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b22c9-68de-472c-b607-347e5ad09c1f None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:31.196826 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b22c9-68de-472c-b607-347e5ad09c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:31.230600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-c22b22c9-68de-472c-b607-347e5ad09c1f None None] MFA Rules not processed for user `53d54e9ab4c74524bf6dd7c86609171e`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:31.231869 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-c22b22c9-68de-472c-b607-347e5ad09c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:31.302064 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1434/2862] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:31.307280 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:31.318984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d2c1d-30ef-4101-a6fa-5d4faaebb288 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:31.324478 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:31.327543 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:31.327900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:31.328267 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:31.328985 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:31.337132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:31.372942 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1429/2863] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:31 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:08:31.374470 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b22c9-68de-472c-b607-347e5ad09c1f None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:31.386321 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b22c9-68de-472c-b607-347e5ad09c1f None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:31.442689 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:31.443299 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:31.443576 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:31.443842 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:31.444507 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:31.447034 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:31.467744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1435/2864] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:31 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:31.471695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:31.482314 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5dec-491e-432e-bfbc-b377e5614432 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:31.486840 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:31.487439 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:31.487718 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:31.487990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:31.488675 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:31.490288 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:31.501119 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1430/2865] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:31.505014 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:31.515399 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a2b7-f3c0-4d00-95e7-7abc14610d35 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:31.520008 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:31.520690 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:31.520973 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:31.521244 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] PATH_INFO: `/v3/projects/2d7e07e47a13436fb9590a480bbcc305/users/b315723610a848688f3576b574142052/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:31.535409 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7e07e47a13436fb9590a480bbcc305, user_id=b315723610a848688f3576b574142052, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:31.538159 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:31.550597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1436/2866] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:31 2022] PUT /identity/v3/projects/2d7e07e47a13436fb9590a480bbcc305/users/b315723610a848688f3576b574142052/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:31.555067 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:31.565117 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5747-e880-4abe-b07f-a8b6ed68b220 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:31.594838 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:31.624073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27673c-dc6f-4945-b726-39a7a0e196af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:31.624711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27673c-dc6f-4945-b726-39a7a0e196af admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:31.624986 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27673c-dc6f-4945-b726-39a7a0e196af admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:31.625252 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27673c-dc6f-4945-b726-39a7a0e196af admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:31.625881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27673c-dc6f-4945-b726-39a7a0e196af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:31.627462 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27673c-dc6f-4945-b726-39a7a0e196af admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:31.637771 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1431/2867] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:31.642300 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:31.652494 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e2484-e348-44a0-b782-cec7d45254ef admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:31.657757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:31.658441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:31.658760 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:31.659030 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] PATH_INFO: `/v3/projects/2d7e07e47a13436fb9590a480bbcc305/users/b315723610a848688f3576b574142052/roles/d175b37bcc59450f897dbc44ba9ac182` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:31.661037 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7e07e47a13436fb9590a480bbcc305, user_id=b315723610a848688f3576b574142052, role_id=d175b37bcc59450f897dbc44ba9ac182)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:31.664498 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:31.675881 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1437/2868] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:31 2022] PUT /identity/v3/projects/2d7e07e47a13436fb9590a480bbcc305/users/b315723610a848688f3576b574142052/roles/d175b37bcc59450f897dbc44ba9ac182 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:32.302240 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afc353-d1f1-4f90-8fc1-a34b9fdf4e97 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:32.302533 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afc353-d1f1-4f90-8fc1-a34b9fdf4e97 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:32.302834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afc353-d1f1-4f90-8fc1-a34b9fdf4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:32.330662 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-05afc353-d1f1-4f90-8fc1-a34b9fdf4e97 None None] MFA Rules not processed for user `b315723610a848688f3576b574142052`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:32.331931 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-05afc353-d1f1-4f90-8fc1-a34b9fdf4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:32.404618 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1432/2869] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:32.415711 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:32.417427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:32.423811 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:32.482233 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db30ac-b9c5-461f-9b22-35330015ac5f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:32.487077 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:32.487694 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:32.487974 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:32.488278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:32.544041 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:32.545511 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:32.554311 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1438/2870] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:33.907432 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afc353-d1f1-4f90-8fc1-a34b9fdf4e97 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:33.917545 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afc353-d1f1-4f90-8fc1-a34b9fdf4e97 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:33.921056 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:33.921646 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:33.921923 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:33.922193 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:33.931611 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:33.933064 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:33.942365 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1433/2871] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:37.908953 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:37.910522 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:37.978738 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef95f16-59b6-4bfa-8a73-8ed9054fea0c service cinder] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:37.981479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:37.981479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:37.981479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:37.981479 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:37.984476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:38.055253 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:38.056884 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:38.067231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1439/2872] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:08:43.938309 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=54 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:47.047938 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:47.049484 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:47.107231 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:47.122029 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=100 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:47.134849 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beef2fa-408c-4f11-a256-af6ad7bb3fb0 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:47.140135 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:47.140774 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.141055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.141325 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.143169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:47.156719 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:47.158313 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:47.169564 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1434/2873] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:47.188538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c498e4a-7ea3-4269-8ff7-4cfbb5bcb1f1 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:47.193142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:47.193807 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.194088 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.194926 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] PATH_INFO: `/v3/users/53d54e9ab4c74524bf6dd7c86609171e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.196080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d54e9ab4c74524bf6dd7c86609171e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:47.198047 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:47.244474 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d54e9ab4c74524bf6dd7c86609171e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.245126 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d54e9ab4c74524bf6dd7c86609171e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.252716 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d54e9ab4c74524bf6dd7c86609171e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.253472 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d54e9ab4c74524bf6dd7c86609171e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.266565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d54e9ab4c74524bf6dd7c86609171e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.278628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1440/2874] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:47 2022] DELETE /identity/v3/users/53d54e9ab4c74524bf6dd7c86609171e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:47.455424 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:47.477916 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473a0fe-2208-4e34-95fc-71e6be80d203 service placement] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:47.541397 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:47.541941 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.542219 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.542507 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] PATH_INFO: `/v3/projects/33ea4b4ef78f4f6ab669f62892410e3a` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.543608 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ea4b4ef78f4f6ab669f62892410e3a)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:47.545552 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:47.591091 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ea4b4ef78f4f6ab669f62892410e3a', 'request_id': 'req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.596617 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ea4b4ef78f4f6ab669f62892410e3a', 'request_id': 'req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.603098 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ea4b4ef78f4f6ab669f62892410e3a', 'request_id': 'req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.614756 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] The token cache is being invalidate because project 33ea4b4ef78f4f6ab669f62892410e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:47.615042 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.615782 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.617910 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1435/2875] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:47 2022] DELETE /identity/v3/projects/33ea4b4ef78f4f6ab669f62892410e3a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:47.622028 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:47.623477 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:47.636441 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e64ff-3f5d-462f-a630-cdbaff1471c1 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:47.693038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:47.693670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.693951 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.694221 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] PATH_INFO: `/v3/users/b315723610a848688f3576b574142052` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.695492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b315723610a848688f3576b574142052)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:47.697521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:47.730384 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:47.746327 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b315723610a848688f3576b574142052'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.746988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b315723610a848688f3576b574142052'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.754378 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b315723610a848688f3576b574142052'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.755020 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b315723610a848688f3576b574142052'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.767119 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-a478438c-3121-43ff-9b04-5f5a2150d338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b315723610a848688f3576b574142052'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:47.779547 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1441/2876] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:08:47 2022] DELETE /identity/v3/users/b315723610a848688f3576b574142052 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:08:47.952725 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:47.954550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:47.969009 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610b21-e2f9-420e-86c3-f8dbaaa6ce8b admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:47.987574 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:48.030401 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.031039 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.031322 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.031598 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] PATH_INFO: `/v3/projects/2d7e07e47a13436fb9590a480bbcc305` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.032745 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d7e07e47a13436fb9590a480bbcc305)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.034669 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.079430 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d7e07e47a13436fb9590a480bbcc305', 'request_id': 'req-dc7e3484-472f-42cd-9a58-2158c96527ce'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:48.085591 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d7e07e47a13436fb9590a480bbcc305', 'request_id': 'req-dc7e3484-472f-42cd-9a58-2158c96527ce'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:48.092024 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d7e07e47a13436fb9590a480bbcc305', 'request_id': 'req-dc7e3484-472f-42cd-9a58-2158c96527ce'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:48.104860 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] The token cache is being invalidate because project 2d7e07e47a13436fb9590a480bbcc305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:08:48.105148 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:48.105758 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-dc7e3484-472f-42cd-9a58-2158c96527ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:08:48.107360 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1436/2877] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:08:47 2022] DELETE /identity/v3/projects/2d7e07e47a13436fb9590a480bbcc305 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:48.112734 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bc63c-7a89-44df-9bfd-f76cc4747b89 None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.113222 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bc63c-7a89-44df-9bfd-f76cc4747b89 None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.113379 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bc63c-7a89-44df-9bfd-f76cc4747b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.133135 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-9b8bc63c-7a89-44df-9bfd-f76cc4747b89 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:48.134343 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9b8bc63c-7a89-44df-9bfd-f76cc4747b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:48.192430 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1442/2878] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:48.201984 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ab51-c0c3-477d-8e01-151e91e339a4 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.202272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ab51-c0c3-477d-8e01-151e91e339a4 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.202547 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ab51-c0c3-477d-8e01-151e91e339a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.223698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-0d62ab51-c0c3-477d-8e01-151e91e339a4 None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:48.225021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-0d62ab51-c0c3-477d-8e01-151e91e339a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:48.232812 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1437/2879] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:48.236969 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bc63c-7a89-44df-9bfd-f76cc4747b89 None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:48.246756 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bc63c-7a89-44df-9bfd-f76cc4747b89 None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:48.251538 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.252199 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.252481 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.252749 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.253390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:08:48.253691 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.255267 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.266977 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1443/2880] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 10 17:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:08:48.271344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62ab51-c0c3-477d-8e01-151e91e339a4 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:48.281236 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62ab51-c0c3-477d-8e01-151e91e339a4 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:48.285864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.286455 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.286727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.287078 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.287711 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.289881 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.315479 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1438/2881] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 10 17:08:48 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:48.319609 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:48.330046 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc346c9-5175-417f-b23c-2464654c7c82 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:48.386565 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.387172 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.387452 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.387726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.388390 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.390328 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.411450 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1444/2882] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 10 17:08:48 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:08:48.415632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:48.425963 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17dad9-6752-403f-a358-e0e6a93ddc00 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:48.433324 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.433934 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.434239 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.434505 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.435144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.436955 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.448037 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1439/2883] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:48.451981 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:48.462595 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9114e8-34b9-4b56-a666-f06eed699fd5 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:48.469670 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c4e90-58eb-4b12-b812-9dd9f63a68c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.470285 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c4e90-58eb-4b12-b812-9dd9f63a68c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.470570 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c4e90-58eb-4b12-b812-9dd9f63a68c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.470839 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c4e90-58eb-4b12-b812-9dd9f63a68c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.471482 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c4e90-58eb-4b12-b812-9dd9f63a68c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.473146 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c4e90-58eb-4b12-b812-9dd9f63a68c0 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.485349 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1445/2884] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 10 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:48.489279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:48.499706 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd674b83-35ad-4b16-bc79-fc18a1d83a46 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:48.504308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.504949 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.505222 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.505489 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] PATH_INFO: `/v3/projects/c02cbefc54534ada8241d42c7f87ba11/users/5db655d2b694453e8ac41eee105482da/roles/230b13c1aa214ed4bb560f74e036a7b7` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.519990 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02cbefc54534ada8241d42c7f87ba11, user_id=5db655d2b694453e8ac41eee105482da, role_id=230b13c1aa214ed4bb560f74e036a7b7)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.522641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.535169 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1440/2885] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 10 17:08:48 2022] PUT /identity/v3/projects/c02cbefc54534ada8241d42c7f87ba11/users/5db655d2b694453e8ac41eee105482da/roles/230b13c1aa214ed4bb560f74e036a7b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:08:48.545451 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.545736 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.546040 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.572320 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:48.575227 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] MFA Rules not processed for user `5db655d2b694453e8ac41eee105482da`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:48.577075 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:48.641947 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1446/2886] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:08:48.652946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:48.654632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:48.706402 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:48.713228 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b582-3c39-425f-90bd-e2cdd7ab2754 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:48.717863 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770e3b2a34d74195bcdf1d0b6f696cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:48.718495 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:48.718776 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:48.719050 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:48.727444 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.728925 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.737571 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1441/2887] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:58.638859 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=26 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:58.733632 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=5 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:01.203039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:01.204496 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:01.272719 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87d0fa-ab74-47d8-8063-22d2b5db159c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:01.277740 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3c0f0-3db5-4172-850e-fb58b718c1ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:01.278347 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c0f0-3db5-4172-850e-fb58b718c1ba service placement] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:01.278626 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c0f0-3db5-4172-850e-fb58b718c1ba service placement] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:01.278901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c0f0-3db5-4172-850e-fb58b718c1ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:01.280652 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9ef3c0f0-3db5-4172-850e-fb58b718c1ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:01.292569 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c0f0-3db5-4172-850e-fb58b718c1ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:01.294165 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c0f0-3db5-4172-850e-fb58b718c1ba service placement] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:01.304549 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1447/2888] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:09:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:09:11.299769 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=32 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:12.301075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:12.363027 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8d938-9afa-4114-878a-c477c4090495 service cinder] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:12.367535 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:12.368175 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:12.368457 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:12.368731 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] PATH_INFO: `/v3/users/5db655d2b694453e8ac41eee105482da` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:12.369834 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db655d2b694453e8ac41eee105482da)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:12.371747 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:12.419138 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db655d2b694453e8ac41eee105482da'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.425946 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db655d2b694453e8ac41eee105482da'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.434541 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db655d2b694453e8ac41eee105482da'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.435160 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db655d2b694453e8ac41eee105482da'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.447468 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db655d2b694453e8ac41eee105482da'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.453301 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1442/2889] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:09:12 2022] DELETE /identity/v3/users/5db655d2b694453e8ac41eee105482da => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:09:12.457828 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] REQUEST_METHOD: `POST` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:12.458278 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:12.458398 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:12.477530 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.auth.core [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] MFA Rules not processed for user `b68f518c53f84700bf1f6ef1bd005525`. Rule list: `[]` (Enabled: `True`). {{(pid=81539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:09:12.478726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:12.537531 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1448/2890] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 10 17:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:09:12.547214 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:12.548655 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:12.605761 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e61622-529b-43ab-89f8-51705e300101 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:12.609423 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:12.610021 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:12.610296 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:12.610578 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:12.620033 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:12.621486 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:12.630727 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1443/2891] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:09:12.895837 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:12.897314 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:12.911038 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887bda-5f15-408e-9e86-97ac4be19a1c None None] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:12.916246 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:12.917462 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:12.917917 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:12.918198 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] PATH_INFO: `/v3/projects/c02cbefc54534ada8241d42c7f87ba11` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:12.919476 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02cbefc54534ada8241d42c7f87ba11)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:12.921276 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:12.969016 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02cbefc54534ada8241d42c7f87ba11', 'request_id': 'req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.974483 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02cbefc54534ada8241d42c7f87ba11', 'request_id': 'req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.981110 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02cbefc54534ada8241d42c7f87ba11', 'request_id': 'req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.994109 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] The token cache is being invalidate because project c02cbefc54534ada8241d42c7f87ba11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:09:12.994523 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.995142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:12.996653 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1449/2892] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:09:12 2022] DELETE /identity/v3/projects/c02cbefc54534ada8241d42c7f87ba11 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:09:22.626634 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:22.990757 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:31.395427 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:31.405336 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764610a-eac5-4b42-b258-246d3a5a8609 service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:31.455754 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:31.456416 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:31.456695 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:31.456964 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:31.458687 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:31.520359 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:31.521799 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:31.532272 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1444/2893] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:09:37.798322 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:37.799948 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:37.815901 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9369a5-d654-4bbd-8c8a-2668e225f1ba admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:37.821606 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:37.822323 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:37.822613 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:37.822894 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] PATH_INFO: `/v3/users/a1c78f05a0f24151a1884faed65d286e` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:37.844220 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c78f05a0f24151a1884faed65d286e)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:37.846177 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:37.891759 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c78f05a0f24151a1884faed65d286e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:37.892435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c78f05a0f24151a1884faed65d286e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:37.899677 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c78f05a0f24151a1884faed65d286e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:37.900365 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c78f05a0f24151a1884faed65d286e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:37.912375 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c78f05a0f24151a1884faed65d286e'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:37.924741 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1450/2894] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:09:37 2022] DELETE /identity/v3/users/a1c78f05a0f24151a1884faed65d286e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:09:38.101247 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:38.111819 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4b91f-22e1-4fa3-99d3-7b911e18f9fb service neutron] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:38.169399 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:38.170029 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:38.170308 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:38.170583 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] PATH_INFO: `/v3/projects/41311d9aa3f54596a61033b0ea82ba80` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:38.181233 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41311d9aa3f54596a61033b0ea82ba80)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:38.183180 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:38.227132 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:38.232209 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41311d9aa3f54596a61033b0ea82ba80', 'request_id': 'req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.237886 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41311d9aa3f54596a61033b0ea82ba80', 'request_id': 'req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.244526 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41311d9aa3f54596a61033b0ea82ba80', 'request_id': 'req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.257778 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] The token cache is being invalidate because project 41311d9aa3f54596a61033b0ea82ba80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:09:38.258093 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.258715 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.260218 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1445/2895] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:09:38 2022] DELETE /identity/v3/projects/41311d9aa3f54596a61033b0ea82ba80 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:09:38.264358 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:38.265761 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:38.279682 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe240a-6b47-4e8b-8dc0-428576aadf81 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:38.338535 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:38.339492 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:38.339928 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:38.340412 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] PATH_INFO: `/v3/users/7bf5bb13c628487f8773438d7f3058e0` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:38.355304 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf5bb13c628487f8773438d7f3058e0)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:38.357280 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:38.402348 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf5bb13c628487f8773438d7f3058e0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.402988 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf5bb13c628487f8773438d7f3058e0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.406487 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:38.414597 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf5bb13c628487f8773438d7f3058e0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.415356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf5bb13c628487f8773438d7f3058e0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.429834 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf5bb13c628487f8773438d7f3058e0'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.442053 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1451/2896] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:09:38 2022] DELETE /identity/v3/users/7bf5bb13c628487f8773438d7f3058e0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:09:38.622641 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:38.624058 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:38.638463 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fa2b5-0d4d-4077-8c5f-8424ed333fa7 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:38.696788 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:38.697512 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:38.697790 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:38.698061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] PATH_INFO: `/v3/projects/b822d87e462e4e3d8f340858ca213ace` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:38.708426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b822d87e462e4e3d8f340858ca213ace)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:38.710344 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:38.755826 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b822d87e462e4e3d8f340858ca213ace', 'request_id': 'req-37503ada-4b72-4f25-b03d-eab3ec77a6c3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.761900 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b822d87e462e4e3d8f340858ca213ace', 'request_id': 'req-37503ada-4b72-4f25-b03d-eab3ec77a6c3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.768279 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b822d87e462e4e3d8f340858ca213ace', 'request_id': 'req-37503ada-4b72-4f25-b03d-eab3ec77a6c3'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.780952 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] The token cache is being invalidate because project b822d87e462e4e3d8f340858ca213ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:09:38.781244 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.781864 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:09:38.783323 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1446/2897] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:09:38 2022] DELETE /identity/v3/projects/b822d87e462e4e3d8f340858ca213ace => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:09:48.438491 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=12,keystone:DELETE=2 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:48.778036 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=63,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:58.847891 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:58.849442 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:58.907071 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156276c-5ceb-41a0-9f4f-7532a4c3f333 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:58.958423 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:58.958954 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:58.959731 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:58.960005 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:58.962039 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:59.034434 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:59.035886 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:59.047144 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1452/2898] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:09:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:10:00.313532 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:00.314974 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:00.328859 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37503ada-4b72-4f25-b03d-eab3ec77a6c3 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:00.332381 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276d73-09b1-490c-93c0-4a8c665ecb8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:00.332979 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276d73-09b1-490c-93c0-4a8c665ecb8e service neutron] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:00.333256 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276d73-09b1-490c-93c0-4a8c665ecb8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:00.333529 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276d73-09b1-490c-93c0-4a8c665ecb8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:00.335223 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-d5276d73-09b1-490c-93c0-4a8c665ecb8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:00.350077 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276d73-09b1-490c-93c0-4a8c665ecb8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:00.351517 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276d73-09b1-490c-93c0-4a8c665ecb8e service neutron] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:00.362717 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1447/2899] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:10:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:10:09.042142 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=74 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:10.357765 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=6 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:11:00.799831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:11:00.801308 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:00.812533 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dab655-9c76-4440-88ba-9db31ac45308 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:11:00.816194 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256af8436dd842fdb4ebe90ec9be26ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:11:00.816800 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:00.817105 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:00.817355 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:00.819025 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:00.828991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:11:00.830411 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:11:00.838568 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1453/2900] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:11:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:11:10.834772 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=6 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:11:22.161592 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] REQUEST_METHOD: `POST` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:22.162075 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:22.162189 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:22.180846 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.auth.core [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] MFA Rules not processed for user `81fba04097094820be201ad24ed2658d`. Rule list: `[]` (Enabled: `True`). {{(pid=81540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:11:22.182043 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:22.236382 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1448/2901] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 10 17:11:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:11:22.278427 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:11:22.279879 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:22.348431 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaded6-c93d-42c6-8e23-0003bb73e5c9 service neutron] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:11:22.353504 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f64c104787604ea4b22e2af2507b3f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:11:22.354104 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:22.354381 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:22.354650 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:22.362700 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:11:22.364303 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:11:22.372155 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1454/2902] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 10 17:11:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:11:28.913625 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:11:28.915059 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:28.969600 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:11:28.971058 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:28.997144 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613e6bd-a5d0-41d4-9c5b-b963698846e5 None None] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:11:29.002445 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:11:29.003051 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] REQUEST_METHOD: `GET` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:29.003329 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:29.003596 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:29.005697 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:29.017206 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:11:29.019055 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:11:29.029073 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1449/2903] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:11:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:11:29.038432 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a7cc-45b1-4caf-87cd-c820cbbf8350 service swift] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:11:29.043744 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:11:29.044555 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:29.044849 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:29.045132 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] PATH_INFO: `/v3/users/226a40e02432469183e0f8b3360126dc` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:29.046435 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226a40e02432469183e0f8b3360126dc)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:11:29.048574 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:11:29.096117 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226a40e02432469183e0f8b3360126dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.097319 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226a40e02432469183e0f8b3360126dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.104098 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226a40e02432469183e0f8b3360126dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.104726 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226a40e02432469183e0f8b3360126dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.117116 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226a40e02432469183e0f8b3360126dc'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.129944 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1455/2904] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:11:28 2022] DELETE /identity/v3/users/226a40e02432469183e0f8b3360126dc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:11:29.300873 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:11:29.312735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b7a35-3c33-44c7-99a9-3cf02eeefbf0 service placement] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:11:29.371841 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:11:29.372426 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:29.372735 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:29.373035 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] PATH_INFO: `/v3/projects/d7d69fb708ed447e8a190efb4be9d5f1` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:29.374149 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d69fb708ed447e8a190efb4be9d5f1)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:11:29.376052 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:11:29.414539 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:11:29.425208 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d69fb708ed447e8a190efb4be9d5f1', 'request_id': 'req-b2d02ba3-e000-4158-9fe8-06364c9d42a1'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.430659 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d69fb708ed447e8a190efb4be9d5f1', 'request_id': 'req-b2d02ba3-e000-4158-9fe8-06364c9d42a1'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.436798 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d69fb708ed447e8a190efb4be9d5f1', 'request_id': 'req-b2d02ba3-e000-4158-9fe8-06364c9d42a1'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.450020 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] The token cache is being invalidate because project d7d69fb708ed447e8a190efb4be9d5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:11:29.450312 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.451114 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.453061 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1450/2905] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:11:29 2022] DELETE /identity/v3/projects/d7d69fb708ed447e8a190efb4be9d5f1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:11:29.456991 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:11:29.458406 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:29.472393 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417836a2-b17f-4547-918c-af58ada50cf7 admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:11:29.477121 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:11:29.534162 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:11:29.534803 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:29.535080 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:29.535356 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] PATH_INFO: `/v3/users/b24535c8baf24b3d8ca6ceff8780238f` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:29.536521 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24535c8baf24b3d8ca6ceff8780238f)` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:11:29.538556 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:11:29.586397 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24535c8baf24b3d8ca6ceff8780238f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.587049 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24535c8baf24b3d8ca6ceff8780238f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.594737 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24535c8baf24b3d8ca6ceff8780238f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.595371 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24535c8baf24b3d8ca6ceff8780238f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.607631 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.notifications [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24535c8baf24b3d8ca6ceff8780238f'} {{(pid=81539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.619539 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1456/2906] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 10 17:11:29 2022] DELETE /identity/v3/users/b24535c8baf24b3d8ca6ceff8780238f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:11:29.788026 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Authenticating user token {{(pid=81540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:11:29.789550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.fernet_utils [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:11:29.802473 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d02ba3-e000-4158-9fe8-06364c9d42a1 admin admin] Validating token access rules against request {{(pid=81540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:11:29.860310 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68f518c53f84700bf1f6ef1bd005525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c8b317543504cf3bafa443bc2186b42', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:11:29.860943 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:11:29.861221 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:11:29.861491 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] PATH_INFO: `/v3/projects/53ed071861ac4e5e8bbaa0f9152c711e` {{(pid=81540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:11:29.862600 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ed071861ac4e5e8bbaa0f9152c711e)` {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:11:29.864550 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] RBAC: Authorization granted {{(pid=81540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:11:29.910253 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ed071861ac4e5e8bbaa0f9152c711e', 'request_id': 'req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.915594 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ed071861ac4e5e8bbaa0f9152c711e', 'request_id': 'req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.921781 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ed071861ac4e5e8bbaa0f9152c711e', 'request_id': 'req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e'} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.934212 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] The token cache is being invalidate because project 53ed071861ac4e5e8bbaa0f9152c711e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:11:29.934698 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.935623 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG keystone.notifications [None req-1aec8d3f-0d3c-43e6-bebd-6ebda4e46c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:11:29.937134 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: [pid: 81540|app: 0|req: 1451/2907] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 10 17:11:29 2022] DELETE /identity/v3/projects/53ed071861ac4e5e8bbaa0f9152c711e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:11:39.616091 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:11:39.930688 dv-cinder-836027-6-iscsi devstack@keystone.service[81540]: DEBUG dbcounter [-] [81540] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:02.389960 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Authenticating user token {{(pid=81539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.391405 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:02.529831 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2eda3-4a6b-4df5-98f0-be76d6a3022f admin admin] Validating token access rules against request {{(pid=81539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.588755 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632df5a-f7ee-46da-b9dd-708a1fd0f85b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836987c3b0a9483db61d8f31ad2af00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '384fce8c44ca4ff4be15fc9495992f7c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.589362 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632df5a-f7ee-46da-b9dd-708a1fd0f85b service placement] REQUEST_METHOD: `GET` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.589639 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632df5a-f7ee-46da-b9dd-708a1fd0f85b service placement] SCRIPT_NAME: `/identity` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.589909 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632df5a-f7ee-46da-b9dd-708a1fd0f85b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.591628 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.fernet_utils [None req-9632df5a-f7ee-46da-b9dd-708a1fd0f85b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:02.603201 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632df5a-f7ee-46da-b9dd-708a1fd0f85b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.604809 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632df5a-f7ee-46da-b9dd-708a1fd0f85b service placement] RBAC: Authorization granted {{(pid=81539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.614514 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: [pid: 81539|app: 0|req: 1457/2908] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:12.610252 dv-cinder-836027-6-iscsi devstack@keystone.service[81539]: DEBUG dbcounter [-] [81539] Writing DB stats keystone:SELECT=89 {{(pid=81539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}